Video object segmentation method based on mask-guided semi-dense contrastive learning

Abstract

The application discloses a video target segmentation method based on mask guided semi-dense contrast learning. The steps of the method include constructing a target encoder and a query encoder, obtaining prototype embeddings of segmentation targets in a video, obtaining pixel-level feature representations of query frames in the video, constructing a loss function of semi-dense contrast learning, setting an updating mode of the target encoder, training the query encoder, dynamically updating a negative sample prototype embedding memory bank, repeatedly training the query encoder, extracting the trained query encoder, extracting image features of a new video in the extracted query encoder, performing pixel affinity matching calculation to obtain a pixel affinity matrix, and performing mask propagation calculation between frames by using the pixel affinity matrix to obtain a segmentation result label. The application avoids the problems of instance discrimination and lack of target spatial focus in video visual representation of semi-supervised video target segmentation.

Description

Technical Field

[0001] This invention belongs to the technical field of computer vision, specifically relating to a video target segmentation method based on mask-guided semi-dense contrastive learning. Background Technology

[0002] Video object segmentation is one of the most important research areas in computer vision. Video object segmentation includes semi-supervised video object segmentation, which can specify the object by giving a mask annotation in the first frame and then perform segmentation on it in subsequent video frames.

[0003] Based on whether labeled information is used during training, existing semi-supervised video object segmentation can be divided into supervised training-based methods and self-supervised training-based methods. Supervised training-based methods are mainly divided into propagation-based methods, matching-based methods, and methods that combine propagation and matching, depending on the solution approach. Self-supervised learning-based semi-supervised video object segmentation generally consists of two stages. The first stage uses unlabeled data to train the model, obtaining a feature extraction model. The second stage inputs video frame images into the feature extraction model trained in the previous stage to obtain the feature representations corresponding to the video frame images. Then, using inter-frame correlation, mask propagation is performed starting from the first frame's ground truth label. The training of the feature extraction model in the first stage requires designing a "self-supervised" signal based on the characteristics of the corresponding original data to complete the network's "pseudo" supervised training. Existing self-supervised learning-based semi-supervised video object segmentation technologies employ generative learning-based or contrastive learning-based methods in the first stage of feature extraction.

[0004] Generative learning-based methods typically generate a result similar to the original data and then construct a loss function between the newly generated and original data. These methods focus more on reconstruction errors in pixel space. Contrastive learning-based methods, on the other hand, do not require designing self-supervised signals by reconstructing the original pixel-level input. Instead, they aim for the network to distinguish different inputs in a higher-order feature space, thereby guiding the network to learn a general feature representation during training and exhibiting stronger generalization ability.

[0005] However, for semi-supervised video object segmentation based on self-supervised training, using generative methods is problematic because the first stage utilizes the inherent attributes of the original data to design auxiliary tasks for training the feature extraction network. The design of these auxiliary tasks can negatively impact the network, often leading to incompatibility when transferring to downstream tasks in the second stage, resulting in decreased segmentation performance. Furthermore, pixel-level reconstruction auxiliary tasks guide the network to implicitly learn correspondences between the image and background, which is redundant in downstream video object segmentation, significantly increasing the computational cost of matching. While contrastive methods, through semantic comparison of instances, can learn more general and versatile feature extraction networks than generative methods, they are mostly based on static images, learning image-level feature representations. Semi-supervised video object segmentation is a dense pixel-level tracking task; using image-level feature representations results in the loss of detailed features of the segmented object and reduces spatial attention to the segmented object, negatively affecting the segmentation results. Summary of the Invention

[0006] To overcome one or more defects and shortcomings of existing technologies, this invention provides a video target segmentation method based on mask-guided semi-dense contrastive learning, which solves the problems of instance discrimination and target space focus in the visual representation of video in semi-supervised video target segmentation tasks by general contrastive learning.

[0007] To achieve the above objectives, the present invention adopts the following technical solution.

[0008] A video object segmentation method based on mask-guided semi-dense contrastive learning includes the following steps:

[0009] S1. Construct the target encoder Query Encoder Obtain the prototype embedding of the segmented target in the video;

[0010] S2. Obtain the pixel-level feature representation of the query frame t in the video;

[0011] S3. Constructing the loss function for semi-dense contrastive learning.

[0012] S4. Set the target encoder The update method;

[0013] S5. Set the values ​​for the training batches, based on the constructed loss function. For query encoder Perform a batch training and dynamically update the target encoder according to the set method. Perform parameter updates; after a single batch of training is completed, perform negative sample prototype embeddings in memory C. kPerform dynamic updates by embedding the prototype of the segmented target obtained in this batch. Update to negative sample prototype embedding memory C k middle;

[0014] S6, For the query encoder Repeat the training process until the set number of training batches is reached, then use the completed query encoder. Extracted and used as a feature extractor for segmentation targets in the video;

[0015] S7, in the extracted query encoder The algorithm extracts image features from a new video. Given the (t-1)th frame and the tth frame, it extracts the corresponding feature representation z. t-1 z t ;

[0016] S8. The obtained feature representation z t-1 z t Perform pixel affinity matching calculations to obtain the pixel affinity matrix A. t,t-1 (i,j);

[0017] S9. Using pixel affinity matrix A t,t-1 (i,j) performs inter-frame mask propagation calculation to obtain the segmentation result label of pixel i in the t-th frame image on the (t-1)-th frame image, thereby segmenting the target from the video.

[0018] Preferably, the prototype embedding of the segmented target in the video is obtained, and the specific process includes:

[0019] S11. Take the first frame image in the video that contains the corresponding segmentation target, perform a set of random data augmentation operations to obtain the augmented view x. k The first frame, with the index k representing the keyframe;

[0020] S12, Enhance the first frame view x k Input to target encoder In the middle, extract the enhanced view x of the first frame. k pixel-level feature representation z k ,and

[0021] Where H, W, and C represent the length, width, and feature dimension of the image, respectively, and θ k Indicates the target encoder The parameter weights, Indicates the enhancement view x k Perform encoding operations;

[0022] S13. Introduce the corresponding labeled mask of the segmentation target in the first frame image as a priori, and calculate... Obtain the global feature embedding c of the segmented target in the video. k As a prototype embedding; where, c k M represents the feature prototype embedding of the current video segmentation target. k This represents the label mask for a given target in the first frame. Indicates z k and M k Perform matrix inner product operations.

[0023] Furthermore, the pixel-level feature representation of the query frame t in the video is obtained, and the specific process includes:

[0024] S21. Perform a set of random data augmentation operations on the t-th frame of the video to obtain the augmented view x. q q represents the meaning of the query;

[0025] S22, Enhance view x q Input to query encoder Obtain its pixel-level feature representation Where, θ q Indicates query encoder The parameter weights, Indicates the enhancement view x q Perform encoding operations.

[0026] Further, query the encoder Target encoder Both models use the DeepLab-v3 model with a ResNet50 network backbone, and the query encoder... Target encoder The weight parameters are not shared.

[0027] Furthermore, the loss function As shown in the following formula:

[0028]

[0029] in, Let represent the feature representation of the i-th pixel in query frame t, where τ is a temperature parameter used to control the smoothness of the output. C k This represents a negative sample prototype embedding memory of length N, and This represents the prototype embedding of the nth negative sample.

[0030] Furthermore, the target encoder The update method is as follows:

[0031] Target encoder The weight parameters are adopted based on the query encoder. The momentum update method for the weight parameters is shown in the following equation:

[0032] θ k ←mθ k +(1-m)θ q

[0033] Where m∈[0,1) is the momentum parameter, and the larger the momentum parameter, the better the target encoder... The slower the update, the more the symbol "←" indicates that the operation on the left is updated based on the momentum on the right.

[0034] Further, in step S6, the query encoder is... The backbone ResNet50 network is extracted and used as a feature extractor.

[0035] Furthermore, the pixel affinity matching calculation is shown in the following formula:

[0036]

[0037] Where i, j∈{1,2,..,hw}, i and j are the pixel points of the t-th frame image and the (t-1)-th frame image, respectively, and z t (i) is the feature vector of pixel i in the t-th frame image, z t-1 (j) is the feature vector of pixel j in the (t-1)th frame image; j′ represents the calculation of pixel i in the tth frame image and all pixels in the (t-1)th frame image.

[0038] Furthermore, the mask propagation calculation process is shown in the following equation:

[0039]

[0040] Among them, y j t-1 Let y represent the segmentation label of pixel j in the (t-1)th frame of the image. i t-1→t This represents the segmentation result label of pixel i in the (t-1)th frame image after the mask propagation calculation. The segmentation result label represents the background or segmentation target in the video frame.

[0041] Furthermore, after obtaining the segmentation results of the target from the new video, the process also includes the following steps:

[0042] To address potential issues such as target deformation, disappearance and reappearance, occlusion, and interference from similar objects in the segmentation results, the mask propagation calculation in step S9 is optimized. Simultaneously, the first frame image and the most recent 20 frames are used for mask propagation calculation, and the labeled mask of the segmented target in the first frame and the predicted segmentation mask of the most recent 20 frames are propagated to the t-th frame image to obtain the final target segmentation result.

[0043] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0044] This invention optimizes the design of a specific task based on a contrastive method and a semi-supervised video object segmentation task setting. It uses the first frame labeled mask given in the task as a guide for contrastive learning, which solves the problems of missing target strength discrimination and missing target focus in the downstream semi-supervised video object segmentation based on the contrastive learning method. It proposes a semi-dense matching contrastive learning method that integrates example-level feature matching and pixel-level feature matching to reduce the amount of computation in the training process and improve training efficiency by utilizing the understanding of the semi-supervised video object segmentation task.

[0045] This invention combines self-supervised contrastive learning with semi-supervised video object segmentation to achieve optimized fusion, making full use of the first frame annotation information provided by the semi-supervised video object segmentation task, thereby reducing the dependence of deep learning model training on annotation data.

[0046] This invention optimizes contrastive learning starting from semi-supervised video object segmentation tasks, and proposes a new semi-dense contrastive learning paradigm guided by object masks. The trained network can learn pixel-level feature representations that are more suitable for semi-supervised video object segmentation tasks without affecting segmentation accuracy.

[0047] This invention requires no additional annotation information, using only the annotation information of the first frame as a mask guide. Training the network requires only a minimal amount of annotation information, resulting in stronger target instance discrimination and target spatial attention. It is better suited for video target segmentation tasks. Furthermore, it includes a negative sample prototype embedding memory C. k It stores example-level feature representations instead of pixel-level feature representations based on generative methods in existing technologies, which greatly alleviates the memory pressure on computers during training.

[0048] This invention overcomes the limitations of labeled data, reduces the huge computational load of pixel-level matching in supervised learning, and speeds up the training process. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating a video target segmentation method based on mask-guided semi-dense contrastive learning according to the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0051] Example

[0052] Combination Figure 1 As shown, the video target segmentation method based on mask-guided semi-dense contrastive learning in this embodiment includes the following steps:

[0053] S1. Construct the target encoder Query Encoder Obtain the prototype embedding of the segmented target in the video; the specific process includes:

[0054] S11. Take the first frame image in the video that contains the corresponding segmentation target, perform a set of random data augmentation operations to obtain the augmented view x. k The first frame, with index k representing a keyframe; this set of random data augmentation operations includes operations performed on the image. One or more data augmentation operations of type, where Data augmentation operations include random flipping, cropping, rotating, grayscale conversion, and color dithering of images;

[0055] S12, Enhance the first frame view x obtained in step S11. k Input to the target encoder In the middle, extract the enhanced view x of the first frame. k pixel-level feature representation z k ,and Where H, W, and C represent the length, width, and feature dimension of the image, respectively, and θ k Indicates the target encoder The parameter weights, Indicates the enhancement view x k Encoding operations are performed to learn the visual feature representation of the input image;

[0056] In this embodiment, a target encoder is preferred. The DeepLab-v3 model with ResNet50 network as the backbone is adopted;

[0057] S13. Introduce the corresponding labeled mask of the segmentation target in the first frame image as a priori, and calculate... Obtain the global feature embedding c of the segmented target in the video. k As a prototype embedding; where, c k M represents the feature prototype embedding of the current video segmentation target. kThis represents the label mask for a given target in the first frame. Indicates z k and M k Perform matrix inner product operations;

[0058] S2. Obtain the pixel-level feature representation of the query frame t in the video; the specific process includes:

[0059] S21. Perform a set of random data augmentation operations on the t-th frame of the video to obtain the augmented view x. q q represents the meaning of the query; data augmentation operations are... type, The types include random image flipping, cropping, rotation, grayscale, and color dithering;

[0060] S22, Enhance view x q Input to query encoder Obtain its pixel-level feature representation Where, θ q Indicates query encoder The parameter weights, Indicates the enhancement view x q Encoding operations are performed to learn the visual feature representation of the input image;

[0061] This embodiment preferably uses a query encoder. The DeepLab-v3 model, with ResNet50 as its backbone, is adopted, and its weight parameters are the same as those of the target encoder. Not shared;

[0062] S3. Construct the loss function for semi-dense contrastive learning;

[0063] Semi-dense contrastive learning is based on the temporal continuity of video frames. The pixel feature representation extracted from one video frame is closer or more similar to the pixel feature representation from another video frame at a different time in the same video in the feature space. That is, all pixels (including background and foreground) in the current frame should have a certain correspondence with the pixels of the segmentation target in the next frame. For example, if the background of the current frame contains semantic information such as rain or sunny, the target visual in the next frame will also show the corresponding features. Considering that the computational cost of pixel-level matching and contrast is huge, and the learning of the correspondence between the background of the previous frame and the background of the current frame does not help much in the segmentation of the target in the current frame, pixel-level and example-level matching and contrastive learning are integrated. The example-level prototype embedding is obtained by mask guidance to replace the pixel-by-pixel matching and contrastive calculation method.

[0064] For the i-th pixel of the image in the query frame t of the video, the corresponding loss function is... As shown in the following formula:

[0065]

[0066] in, Let represent the feature representation of the i-th pixel in query frame t, where τ is a temperature parameter used to control the smoothness of the output. C k This represents a negative sample prototype embedding memory of length N, and Let C represent the prototype embedding of the nth negative sample. k Prototype embeddings of segmented targets obtained during the training process are stored as negative samples for contrastive learning;

[0067] By matching pixel-level feature representations with example-level prototype embeddings, the feature extraction network is optimized to maximize the pixel feature representation. and the prototype embedding of the segmentation target Consistency, while increasing the corresponding instances of segmented targets in other videos. From the perspective of feature space, the differences between them can be understood as bringing pixel-level feature representations closer together. And the segmentation of the target prototype embedding in the same video The feature distance in the feature space is used to cluster them together, and then the feature representations are spaced as far apart as possible. Feature distance between the prototype embeddings of other video target instances;

[0068] Using a labeled mask as a priori guide, the loss function... The calculation results are sent back to the query encoder. Optimization in this process can improve the query encoder. Extracted feature representation It has stronger target instance space attention and target instance discrimination ability;

[0069] S4. Set the target encoder The update method;

[0070] Due to the query encoder Parameters are updated frequently during each batch of training. To ensure that the negative sample prototype embedding memory C is maintained... k Negative sample prototype embeddings stored in To ensure stability within the same feature space and stabilize the distance calculation between negative samples, the target encoder should... The weighting parameters are adopted according to The momentum update method is shown in the following equation:

[0071] θ k ←mθ k +(1-m)θq

[0072] Where m∈[0,1) is the momentum parameter, and the larger the momentum parameter, the better the target encoder... The slower the update, the more the symbol "←" indicates that the left-side operation is updated based on the right-side momentum to obtain a new target encoder. Weighting parameter θ k ;

[0073] S5. Set the values ​​for the training batches, based on the constructed loss function. For query encoder Perform a batch training and dynamically update the target encoder according to the set method. Perform parameter updates; due to the target encoder Slow updates, with negative sample prototype embedding memory C updated after each batch of training. k Perform dynamic updates by embedding the prototype of the segmented target obtained in this batch. Update to negative sample prototype embedding memory C k In this process, the prototype embedding of the segmented target is... Relative to the next batch of new input videos, it becomes a negative sample prototype embedding. Embed the prototype of the negative sample from the older training batch Deletion is performed to ensure that the negative sample prototype is embedded in memory C. k All prototype embeddings As much as possible, within the same feature space;

[0074] S6, For the query encoder Repeat the training process and update the target encoder synchronously. The query encoder will be trained until the set number of training batches is reached. Extracted and used as a feature extractor for segmentation targets in the video;

[0075] This embodiment preferably uses a query encoder. The backbone ResNet50 network is used as a feature extractor in subsequent steps;

[0076] S7. Input the new video into the query encoder extracted in step S6. Image feature extraction is performed, and a query encoder with fixed parameters is trained. Given the (t-1)th frame image and the tth frame image, perform feature extraction to obtain the corresponding feature representation z. t-1 z t ;

[0077] S8. The obtained feature representation z t-1 zt Perform pixel affinity matching calculations as shown in the following formula to obtain the pixel affinity matrix A. t,t-1 (i,j):

[0078]

[0079] At this point, i, j ∈ {1, 2, ..., hw}, where i and j are the pixel points of the t-th frame and the (t-1)-th frame, respectively, and z t (i) is the feature vector of pixel i in the t-th frame image, z t-1 (j) is the feature vector of pixel j in the (t-1)th frame image, hw represents the upper limit of the number of pixels; j′ represents the meaning of accumulating the pixel i of the tth frame image and all pixels of the (t-1)th frame image in the denominator.

[0080] S9. Utilize the correlation between video frames, using the pixel affinity matrix A t,t-1 (i,j) performs inter-frame mask propagation calculation to obtain the segmentation result label of pixel i in frame t in frame (t-1), thereby segmenting the target from the video. The mask propagation calculation process is shown in the following formula:

[0081]

[0082] Among them, y j t-1 Let y represent the segmentation label of pixel j in the (t-1)th frame of the image. i t-1→t This represents the segmentation result label of pixel i in the t-th frame image after the mask propagation calculation. The segmentation result label specifically refers to the background or segmentation target in the video frame.

[0083] In this preferred embodiment, to improve local attention, only the labels of pixel j in the first 10 most similar (t-1)th frame images are selected for mask propagation calculation of pixel i in the tth frame image;

[0084] S10. For difficult scenarios that may exist in step S9, such as drastic deformation of the segmented target, target disappearance and reappearance, target occlusion, and interference from similar objects, the calculation formula in step S9 is optimized according to the principles of propagation and matching. At the same time, the first frame image and the most recent 20 frames image are used to perform mask propagation calculation. The labeled mask of the segmented target in the first frame and the predicted segmentation mask of the most recent 20 frames are propagated to the t-th frame image to obtain the final target segmentation result.

[0085] Compared with the prior art, the beneficial effects of this embodiment are as follows:

[0086] This embodiment optimizes the design of a specific task based on the contrastive method and the semi-supervised video object segmentation task setting. The first frame labeled mask given in the task is used as the guide for contrastive learning. It solves the problems of missing target strength discrimination and missing target focus in the downstream semi-supervised video object segmentation based on the contrastive learning method. It proposes a semi-dense matching contrastive learning method that integrates example-level feature matching and pixel-level feature matching to reduce the amount of computation in the training process and improve training efficiency by utilizing the understanding of the semi-supervised video object segmentation task.

[0087] This embodiment combines the self-supervised contrastive learning approach with semi-supervised video object segmentation to achieve optimized fusion. It makes full use of the first frame annotation information provided by the semi-supervised video object segmentation task, reducing the dependence of the deep learning model training process on the annotation data.

[0088] This embodiment optimizes contrastive learning starting from the semi-supervised video object segmentation task, and proposes a new semi-dense contrastive learning paradigm guided by object masks. The trained network can learn pixel-level feature representations that are more suitable for the semi-supervised video object segmentation task without affecting the segmentation accuracy.

[0089] This embodiment requires no additional annotation information, using only the annotation information of the first frame as a mask guide. Training the network requires only a minimal amount of annotation information, resulting in stronger target instance discrimination and target spatial attention. It is better suited for video target segmentation tasks. Furthermore, the negative sample prototype embedding memory C... k It stores example-level feature representations instead of pixel-level feature representations based on generative methods in existing technologies, which greatly alleviates the memory pressure on computers during training.

[0090] This embodiment overcomes the limitations of labeled data, reduces the huge computational load of pixel-level matching in supervised learning, and speeds up the training process.

[0091] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A video target segmentation method based on mask-guided semi-dense contrastive learning, characterized in that, The steps include the following: S1. Construct the target encoder Query Encoder θ k Indicates the target encoder The parameter weights, θ q Indicates query encoder The parameter weights are used to obtain the prototype embedding of the segmented target in the video; S2. Obtain the pixel-level feature representation of the query frame t in the video; S3. Constructing the loss function for semi-dense contrastive learning. S4. Set the target encoder The update method; S5. Set the values ​​for the training batches, based on the constructed loss function. For query encoder Perform a batch training and dynamically update the target encoder according to the set method. Perform parameter updates; after a single batch of training is completed, perform negative sample prototype embeddings in memory C. k Perform dynamic updates by embedding the prototype of the segmented target obtained in this batch. Update to negative sample prototype embedding memory C k middle; S6, For the query encoder Repeat the training process until the set number of training batches is reached, then use the completed query encoder. Extracted and used as a feature extractor for segmentation targets in the video; S7, in the extracted query encoder The algorithm extracts image features from a new video. Given the (t-1)th frame and the tth frame, it extracts the corresponding feature representation z. t-1 z t ; S8. The obtained feature representation z t-1 z t Perform pixel affinity matching calculations to obtain the pixel affinity matrix A. t,t-1 (i,j), where i and j are the pixels of the t-th frame and the (t-1)-th frame, respectively; S9. Using pixel affinity matrix A t,t-1 (i,j) performs inter-frame mask propagation calculation to obtain the segmentation result label of pixel i in the t-th frame image on the (t-1)-th frame image, thereby segmenting the target from the video.

2. The video target segmentation method based on mask-guided semi-dense contrastive learning according to claim 1, characterized in that, The specific process of obtaining the prototype embedding of the segmented target in the video includes: S11. Take the first frame image in the video that contains the corresponding segmentation target, perform a set of random data augmentation operations to obtain the augmented view x. k The first frame, with the index k representing the keyframe; S12, Enhance view x k The first frame is input to the target encoder In the middle, extract the enhanced view x k The pixel-level feature representation of the first frame z k ,and Where H, W, and C represent the length, width, and feature dimension of the image, respectively, and θ k Indicates the target encoder The parameter weights, Indicates the enhancement view x k Perform encoding operations; S13. Introduce the corresponding labeled mask of the segmentation target in the first frame image as a priori, and calculate... Obtain the global feature embedding c of the segmented target in the video. k As a prototype embedding; where, c k M represents the global feature prototype embedding of the current video segmentation target. k This represents the label mask for a given target in the first frame. Indicates z k and M k Perform matrix inner product operations.

3. The video target segmentation method based on mask-guided semi-dense contrastive learning according to claim 2, characterized in that, The specific process of obtaining the pixel-level feature representation of the query frame t in the video includes: S21. Perform a set of random data augmentation operations on the image of the query frame t of the video to obtain the augmented view x. q q represents the meaning of the query; S22, Enhance view x q Input to query encoder Obtain its pixel-level feature representation Where, θ q Indicates query encoder The parameter weights, Indicates the enhancement view x q Perform encoding operations.

4. The video target segmentation method based on mask-guided semi-dense contrastive learning according to claim 3, characterized in that, The query encoder Target encoder Both models use the DeepLab-v3 model with a ResNet50 network backbone, and the query encoder... Target encoder The weight parameters are not shared.

5. The video target segmentation method based on mask-guided semi-dense contrastive learning according to claim 4, characterized in that, The loss function As shown in the following formula: in, Let represent the feature representation of the i-th pixel in query frame t, where τ is a temperature parameter used to control the smoothness of the output. C k This represents a negative sample prototype embedding memory of length N, and This represents the prototype embedding of the nth negative sample.

6. The video target segmentation method based on mask-guided semi-dense contrastive learning according to claim 5, characterized in that, The target encoder The update method is as follows: Target encoder The weight parameters are adopted based on the query encoder. The momentum update method for the weight parameters is shown in the following equation: i k ←mθ k +(1-m)θ q Where m∈[0,1) is the momentum parameter, and the larger the momentum parameter, the better the target encoder... The slower the update, the more the symbol "←" indicates that the operation on the left is updated based on the momentum on the right.

7. The video target segmentation method based on mask-guided semi-dense contrastive learning according to claim 6, characterized in that, In step S6, the query encoder is... The backbone ResNet50 network is extracted and used as a feature extractor.

8. The video target segmentation method based on mask-guided semi-dense contrastive learning according to claim 7, characterized in that, The pixel affinity matching calculation is shown in the following formula: Where i, j∈{1,2,..,hw}, i and j are the pixel points of the t-th frame image and the (t-1)-th frame image, respectively, and z t (i) is the feature vector of pixel i in the t-th frame image, z t-1 (j) is the feature vector of pixel j in the (t-1)th frame image, and hw represents the upper limit of the number of pixels; j ′ This means that the pixel i in the t-th frame image is calculated together with all pixels in the (t-1)-th frame image.

9. The video target segmentation method based on mask-guided semi-dense contrastive learning according to claim 8, characterized in that, The process of mask propagation calculation is shown in the following formula: Among them, y j t-1 Let y represent the segmentation label of pixel j in the (t-1)th frame of the image. i t-1→t This represents the segmentation result label of pixel i in the (t-1)th frame image after the mask propagation calculation. The segmentation result label represents the background or segmentation target in the video frame.

10. The video target segmentation method based on mask-guided semi-dense contrastive learning according to claim 9, characterized in that, After obtaining the segmentation results of the target from the new video, the following steps are also included: To address potential issues such as target deformation, disappearance and reappearance, occlusion, and interference from similar objects in the segmentation results, the mask propagation calculation in step S9 is optimized. Simultaneously, the first frame image and the most recent 20 frames are used for mask propagation calculation, and the labeled mask of the segmented target in the first frame and the predicted segmentation mask of the most recent 20 frames are propagated to the t-th frame image to obtain the final target segmentation result.

Images