Weakly supervised video target segmentation method and device based on border annotation

Abstract

The present invention discloses a weakly supervised video target segmentation method and device based on bounding box annotation. The method includes the following steps: training a pseudo-annotation generation model based on the PReMVOS model on an image segmentation dataset; using the pseudo-annotation generation model to generate corresponding pseudo-mask annotations for the video data and bounding box annotations frame by frame; and using the generated pseudo-mask annotations to train a video target segmentation model using a "cooperative teaching" algorithm. This method uses low-cost bounding box annotations to train the video target segmentation model, thereby reducing the cost of annotating new data when migrating the video target segmentation model to actual application scenarios and reducing the difficulty of implementing the video target segmentation model. By using the "cooperative teaching" training algorithm, the existing large number of video target tracking datasets can be more fully utilized to train the video target segmentation model, thereby enhancing the performance and generalization ability of the video target segmentation model.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a weakly supervised video target segmentation method and device based on border annotation. Background Art

[0002] Video object segmentation involves isolating specific objects or objects of human interest within each frame of a given video. As a fundamental task in computer vision, video object segmentation is crucial for video understanding and analysis. In terms of specific applications, video object segmentation is crucial for video editing, human-computer interaction, autonomous driving, and other practical applications.

[0003] Existing advanced algorithms and techniques are primarily based on deep learning. With the availability of large-scale, finely annotated datasets such as DAVIS and YouTube-VOS, existing annotated data can essentially support the training of deep learning models, and some existing methods have already achieved impressive results. Existing video object segmentation algorithms primarily include matching-based algorithms, possible object-based algorithms, mask propagation-based algorithms, and object tracking-based algorithms. However, model training in these algorithms relies heavily on large-scale, high-quality annotated data from existing datasets. Existing video object segmentation techniques require fine, pixel-level mask annotations for each frame during training. This is prohibitively expensive, limiting the scale of datasets and making the translation of existing methods to specific application scenarios unacceptable. Currently, methods that achieve good performance rely heavily on large amounts of high-quality, finely annotated data, meaning they are largely data-driven. The limited size of datasets, however, has limited the development of these data-driven deep learning algorithms. Summary of the Invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and provide a weakly supervised video target segmentation method and device based on border annotation. This method uses low-cost border annotation to train the video target segmentation model, thereby reducing the cost of annotating new data when the video target segmentation model is migrated to actual application scenarios, and reducing the difficulty of implementing the video target segmentation model; by using a "cooperative teaching" training algorithm, it is possible to make more full use of the existing large number of video target tracking data sets to train the video target segmentation model, thereby enhancing the performance and generalization ability of the video target segmentation model.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] On the one hand, the present invention provides a weakly supervised video object segmentation method based on border annotation, comprising the following steps:

[0007] A pseudo-annotation generation model based on the PReMVOS model is trained on the image segmentation dataset. The input of the pseudo-annotation generation model is the original video and the corresponding bounding box annotation, and the output is the pseudo mask annotation;

[0008] Use the pseudo-annotation generation model to generate corresponding pseudo-mask annotations for the video data and bounding box annotations frame by frame;

[0009] The generated pseudo-mask annotations are used to train a video target segmentation model using a "cooperative teaching" algorithm, and the trained video target segmentation model is used to perform target segmentation on the video data to obtain a target segmentation result. The "cooperative teaching" algorithm is to filter out cleaner data for each other for training in each iteration of the training phase for two networks with the same structure but different parameters, thereby alleviating the impact of noisy annotations.

[0010] As a preferred technical solution, the pseudo-annotation generation model based on the PReMVOS model is trained on the image segmentation dataset, which refers to designing a pseudo-annotation generation model using the optimization refinement module in the PReMVOS model and training it on the Mapillary dataset of the image segmentation dataset, specifically:

[0011] The input of the pseudo-annotation generation model is the original image and the corresponding bounding box annotation. The original image and the binary image corresponding to the bounding box annotation are concatenated to obtain a four-channel original input. On the Mapillary dataset, the bounding box annotation is inferred based on the original fine mask annotation in the dataset. That is, for each mask annotation, the bounding box that just completely contains the mask is used as the corresponding bounding box annotation. The pixel value within the bounding box of the binary image is 1, and the rest is 0.

[0012] The marked border is appropriately enlarged to obtain a cropping area, and the original input is cropped according to the cropping area to obtain the cropped image;

[0013] The cropped image is input into the segmentation network to output the segmentation mask corresponding to the target object.

[0014] As a preferred technical solution, the specific method for appropriately enlarging the marked border is as follows:

[0015] Extend the labeled border by n pixels in the four directions of top, bottom, left, and right.

[0016] As a preferred technical solution, the segmentation network structure used in the pseudo-annotation generation model is the Deeplab-v3+ structure, and the pixel-by-pixel cross entropy function is used as the loss function for model training.

[0017] As a preferred technical solution, the pseudo-annotation generation model is used to generate corresponding pseudo-mask annotations for the video data and the bounding box annotations frame by frame, specifically:

[0018] Use the video-to-image conversion tool to convert the video data in the Youtube-VOS dataset into image frames;

[0019] Input each image frame and its corresponding bounding box annotation into the pseudo-annotation generation model to obtain the pseudo mask annotation of each frame;

[0020] For multiple target objects in the same frame, each object is processed one by one to obtain a pseudo mask annotation for each object when generating a pseudo mask annotation. If two pseudo mask annotations overlap, the overlapping part is considered to belong to the pseudo mask annotation with a smaller area.

[0021] As a preferred technical solution, the ffmpeg tool is used to convert video data into image frames.

[0022] As a preferred technical solution, the generated pseudo mask annotation is used to train the video target segmentation model using the "cooperative teaching" algorithm, specifically:

[0023] We randomly initialize two video object segmentation models with the same structure but different parameters, denoted as Model A and Model B. We then use the generated pseudo-mask annotations on the YouTube-VOS dataset and perform preliminary training on Models A and B using a frame-by-frame and pixel-by-pixel cross-entropy loss function. The training process ensures that Models A and B use different training samples in each training iteration.

[0024] After completing the preliminary training, the "cooperative teaching" training phase begins. In each training iteration, a batch of training samples are sampled, and the training samples are segmented once using models A and B respectively. The frame-by-frame and pixel-by-pixel cross entropy loss function values ​​between the segmentation results and the generated pseudo mask annotations are calculated. According to the output results of model A, the cross entropy loss function values ​​of the pixels in the border annotation area are sorted, and the pixels with the smallest R(T)% loss function values ​​are selected, and the pseudo mask corresponding to the pixels is used to annotate the training model B. Where T is the current training iteration number, T k is the parameter that controls the rate of increase of R(T), and τ is the parameter that controls the maximum value of R(T);

[0025] According to the output of model B, the cross entropy loss function values ​​of the pixels in the bounding box annotation area are sorted, and the pixels with the smallest loss function value R(T)% are selected. The pseudo mask corresponding to the pixels is used to annotate the training model A.

[0026] For pixels outside the bounding box, all are treated as background categories to train models A and B;

[0027] When the prediction results of model A and model B gradually converge, the training ends and model A is taken as the final video target segmentation model.

[0028] As a preferred technical solution, the R(T)% parameter gradually increases with the increase in the number of training iterations, that is, more and more pseudo-mask annotations will be screened out during the training process to participate in the training model.

[0029] On the other hand, the present invention provides a weakly supervised video target segmentation system based on border annotation, which is applied to the weakly supervised video target segmentation method based on border annotation, including a pseudo-annotation generation model training module, a pseudo-mask annotation generation module and a segmentation model training module;

[0030] The pseudo-annotation generation model training module is used to train a pseudo-annotation generation model based on the PReMVOS model on an image segmentation dataset;

[0031] The pseudo mask annotation generation module is used to generate corresponding pseudo mask annotations for the video data and the bounding box annotations frame by frame using the pseudo annotation generation model;

[0032] The segmentation model training module is used to utilize the generated pseudo mask annotations and use a "cooperative teaching" algorithm to train the video object segmentation model.

[0033] Another aspect of the present invention provides a computer-readable storage medium comprising a program, which, when executed, implements the weakly supervised video object segmentation method based on border annotation.

[0034] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0035] In response to the problems in the field of video target segmentation, such as high cost of training data annotation and difficulty in collecting sufficiently large-scale data, the present invention proposes a weakly supervised video target segmentation method and device based on bounding box annotation. This method trains the model using only bounding box annotation, and the performance can reach more than 90% of the performance of the model trained using fine mask annotation.

[0036] Since this method can make full use of low-cost bounding box annotation to train the video target segmentation model, it can greatly reduce the annotation cost during the training of the video target segmentation model, thereby reducing the cost of migrating the video target segmentation model to actual application scenarios and reducing its implementation difficulty.

[0037] There are two main reasons why this method can successfully use bounding box annotations to train video target segmentation models: first, the pseudo-annotation generation model in the method can generate relatively high-quality pseudo-annotations. These pseudo-annotations contain rich structured information that is beneficial to the training of video target segmentation models; second, the training algorithm based on "cooperative teaching" can greatly alleviate the impact of noise in pseudo-annotations on training, thereby preventing the model from being excessively disturbed by erroneous pseudo-annotations. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 Flowchart of a weakly supervised video object segmentation method based on border annotation according to an embodiment of the present invention;

[0040] Figure 2 Schematic diagram of a data pseudo-annotation generation model according to an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the "cooperative teaching" training algorithm of an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of the structure of a weakly supervised video object segmentation system based on border annotation according to an embodiment of the present invention;

[0043] Figure 5 Schematic diagram of the structure of the storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments and drawings in this application. It should be understood that the drawings are only for illustrative purposes and cannot be understood as limiting this patent. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0045] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it mean that each embodiment is an independent or alternative embodiment that is mutually exclusive with other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0046] Example

[0047] like Figure 1 As shown, this embodiment is a weakly supervised video target segmentation method based on border annotation, which includes the following steps:

[0048] Step 1: Use the image segmentation dataset to train a pseudo-annotation generation model based on the PReMVOS model, specifically:

[0049] The present invention uses the refinement module in the PReMVOS model to design a pseudo-annotation generation model and trains the model on the Mapillary dataset. The pseudo-annotation generation model in the present invention takes the original image and the corresponding bounding box annotation as input and outputs the corresponding mask annotation. The specific schematic diagram of the model is shown in the figure. Figure 2 shown.

[0050] The main idea of ​​this model is to use bounding box annotations as an input channel of the deep neural network, so that the model can obtain some information about the location of the target object and thus segment the input image. At the same time, based on the bounding box annotations, the image area that is not related to the target object can be cropped out to allow the model to focus on the segmentation of the target object.

[0051] The model's details are as follows: The model's input is the original image and its corresponding bounding box annotations. First, the original image and the binary image corresponding to the bounding box annotations (the pixel value inside the bounding box is 1, and the rest is 0) are concatenated to obtain a four-channel original input. The bounding box is then appropriately enlarged, extending 50 pixels in each of the four directions to create a cropped region. The original input is then cropped according to the cropped region to obtain a cropped image, which serves as the input to a segmentation network. The segmentation network outputs a segmentation mask corresponding to the target object.

[0052] The segmentation network structure used in this model is the Deeplab-v3+ structure, and the pixel-by-pixel cross entropy function is used as the loss function for model training to obtain a pseudo-annotation generation model.

[0053] Step 2: Use the pseudo-annotation generation model to generate corresponding pseudo-mask annotations for the video data and bounding box annotations frame by frame. Specifically:

[0054] First, use a video image conversion tool (such as ffmpeg) to convert the video data in the Youtube-VOS dataset into image frames. Then, for each image frame, it and the corresponding bounding box annotation are input into the pseudo-annotation generation model to obtain a pseudo-mask annotation for each frame. Since there may be multiple target objects that need to be segmented in the same frame, the bounding boxes of these objects may overlap. For this situation where there are multiple target objects in the same frame, each object is processed one by one when generating the pseudo-mask annotation to obtain a pseudo-mask annotation for each object. If two pseudo-mask annotations overlap, the overlapping part is considered to belong to the pseudo-mask annotation with a smaller area.

[0055] Step 3: Using the generated pseudo-mask annotations, use the "cooperative teaching" algorithm to train the video object segmentation model, specifically:

[0056] This paper modifies the "co-teaching" algorithm in the field of learning with noisy annotations and applies it to the learning of video object segmentation models, thereby alleviating the negative impact of pseudo-mask annotations with a certain amount of noise on the model learning process.

[0057] The following describes the "cooperative teaching" algorithm and the modifications made by the present invention to the algorithm:

[0058] The details of the "cooperative teaching" algorithm are as follows Figure 3 As shown in the figure, in the "cooperative teaching" algorithm, two networks with the same structure but different parameters filter out cleaner data for each other's training in each iteration of the training phase, thereby alleviating the impact of noisy labels. The principle of this algorithm is that deep neural networks tend to "remember" easy-to-learn samples first and "remember" difficult-to-learn samples later during the training phase. The distribution of noisy labeled samples is usually difficult to learn. Therefore, during the training phase, the loss function value can be used to distinguish between noisy labeled samples and clean samples to a certain extent, thereby screening a certain proportion of relatively clean samples for model training.

[0059] In the present invention, the modified "cooperative teaching" algorithm process is as follows:

[0060] First, we randomly initialize two video object segmentation models with the same structure but different parameters, denoted as Model A and Model B. Using the generated pseudo-mask annotations on the YouTube-VOS dataset, we perform a preliminary training of Models A and B for a small number of iterations using a frame-by-frame and pixel-by-pixel cross-entropy loss function. The training process ensures that Models A and B use different training samples in each training iteration to ensure a certain degree of differentiation between Models A and B.

[0061] After completing the initial training, the next step is the "cooperative teaching" training phase. In each training iteration, a batch of training samples are sampled and segmented using models A and B respectively. The frame-by-frame and pixel-by-pixel cross entropy loss function values ​​between the segmentation results and the generated pseudo mask annotations are calculated. For the output results of model A, the cross entropy loss function values ​​of the pixels within the bounding box annotation area are sorted, and the pixels with the smallest R(T)% loss function values ​​are selected to train model B using the pseudo annotations corresponding to the pixels. Where T is the current training iteration number, T k is the parameter that controls the rate of increase of R(T), and τ is the parameter that controls the maximum value of R(T);

[0062] For the output of model B, the same operation is performed to filter out the pixels with the smallest loss function value R(T)% in the bounding box annotation area, and the corresponding pseudo-masks are used to train model A. The parameter R(T)% gradually increases with the number of training iterations, that is, more and more pseudo-annotations will be selected during the training process to participate in the model training.

[0063] For pixels outside the bounding box, all are treated as background categories to train models A and B. In the later stages of the entire training process, the prediction results of models A and B will gradually converge. Therefore, after the training is completed, model A can be used as the final video target segmentation model. The model trained in this way will have a better ability to segment target objects in the video.

[0064] The "cooperative teaching" algorithm in this embodiment specifically uses the AGAME video target segmentation model, but any other model that can use pixel-by-pixel cross entropy as the loss function is applicable to this method; in the experiment, only bounding box annotations with low annotation costs were used on the Youtube-VOS dataset, and the pseudo-annotation generation model trained by the present invention was used. The "cooperative teaching" algorithm was used to train the AGAME video target segmentation model, and a performance of 59.5% mIoU (mean intersection over union) was achieved, reaching 90.6% of the performance of the model trained using fine mask annotations (65.7% mIoU), which verifies the effectiveness and practicality of the weakly supervised video target segmentation method based on bounding box annotations described in the present invention.

[0065] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0066] Based on the same idea as the weakly supervised video target segmentation method based on border annotation in the above embodiment, the present invention also provides a weakly supervised video target segmentation system based on border annotation, which can be used to perform the above-mentioned weakly supervised video target segmentation method based on border annotation. For ease of explanation, in the structural diagram of the weakly supervised video target segmentation system based on border annotation in this embodiment, only the parts related to the embodiment of the present invention are shown. Those skilled in the art will understand that the illustrated structure does not constitute a limitation of the device, and it may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0067] like Figure 4 As shown, in another embodiment of the present application, a weakly supervised video object segmentation system 100 based on border annotation is provided, which includes a pseudo annotation generation model training module 101, a pseudo mask annotation generation module 102 and a segmentation model training module 103;

[0068] The pseudo-annotation generation model training module 101 is used to train a pseudo-annotation generation model based on the PReMVOS model on an image segmentation dataset;

[0069] The pseudo mask annotation generation module 102 is used to generate corresponding pseudo mask annotations for the video data and the bounding box annotations frame by frame using the pseudo annotation generation model;

[0070] The segmentation model training module 103 is used to train the video object segmentation model using the generated pseudo mask annotations using a “cooperative teaching” algorithm.

[0071] It should be noted that the weakly supervised video target segmentation system based on border annotation of the present invention corresponds one-to-one to the weakly supervised video target segmentation method based on border annotation of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the weakly supervised video target segmentation method based on border annotation are applicable to the embodiment of the weakly supervised video target segmentation system based on border annotation. For specific contents, please refer to the description in the embodiment of the method of the present invention. No further details will be given here. This is hereby declared.

[0072] In addition, in the implementation of the weakly supervised video target segmentation system based on border annotation in the above-mentioned embodiment, the logical division of each program module is only an example. In actual application, the above-mentioned functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the weakly supervised video target segmentation system based on border annotation is divided into different program modules to complete all or part of the functions described above.

[0073] like Figure 5As shown, in another embodiment, a computer-readable storage medium 200 is further provided, which stores a program 201 in a memory. When the program 201 is executed by the processor 202, a weakly supervised video object segmentation method based on border annotation is implemented, specifically:

[0074] Train a pseudo-annotation generation model based on the PReMVOS model on an image segmentation dataset;

[0075] Use the pseudo-annotation generation model to generate corresponding pseudo-mask annotations for the video data and bounding box annotations frame by frame;

[0076] The generated pseudo-mask annotations are used to train a video object segmentation model using a "co-teaching" algorithm.

[0077] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-described methods. In particular, any reference to memory, storage, database, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory.

[0078] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A weakly supervised video target segmentation method based on bounding box annotation, characterized by: The steps include: A pseudo-annotation generation model based on the PReMVOS model is trained on the image segmentation dataset. The input of the pseudo-annotation generation model is the original video and the corresponding bounding box annotation, and the output is the pseudo mask annotation; Use the pseudo-annotation generation model to generate corresponding pseudo-mask annotations for the video data and bounding box annotations frame by frame; The generated pseudo-mask annotations are used to train a video object segmentation model using a "cooperative teaching" algorithm. The trained video object segmentation model is then used to segment the video data to obtain object segmentation results. The "cooperative teaching" algorithm involves two networks with the same structure but different parameters filtering out cleaner data for each other during each iteration of the training phase, thereby mitigating the impact of noisy annotations. The generated pseudo mask annotation is used to train the video object segmentation model using the "cooperative teaching" algorithm, specifically: We randomly initialize two video object segmentation models with the same structure but different parameters, denoted as Model A and Model B. We then use the generated pseudo-mask annotations on the YouTube-VOS dataset and perform preliminary training on Models A and B using a frame-by-frame and pixel-by-pixel cross-entropy loss function. The training process ensures that Models A and B use different training samples in each training iteration. After completing the preliminary training, the "cooperative teaching" training phase begins. In each training iteration, a batch of training samples are sampled and segmented using models A and B respectively. The frame-by-frame and pixel-by-pixel cross entropy loss function values ​​between the segmentation results and the generated pseudo mask annotations are calculated. According to the output of model A, the cross entropy loss function values ​​of the pixels within the bounding box annotation area are sorted, and the pixels with the smallest R(T)% loss function values ​​are selected. The pseudo mask corresponding to the pixels is used to annotate the training model B. Where T is the current training iteration number, T k is the parameter that controls the rate of increase of R(T), and τ is the parameter that controls the maximum value of R(T); According to the output of model B, the cross entropy loss function values ​​of the pixels in the bounding box annotation area are sorted, and the pixels with the smallest loss function value R(T)% are selected. The pseudo mask corresponding to the pixels is used to annotate the training model A. For pixels outside the bounding box, all are treated as background categories to train models A and B; When the prediction results of model A and model B gradually converge, the training ends and model A is taken as the final video target segmentation model.

2. The weakly supervised video object segmentation method based on border annotation according to claim 1 is characterized in that The training of a pseudo-annotation generation model based on the PReMVOS model on an image segmentation dataset refers to designing a pseudo-annotation generation model using the optimization refinement module in the PReMVOS model and training it on the Mapillary dataset, specifically: The input of the pseudo-annotation generation model is the original image and the corresponding bounding box annotation. The original image and the binary image corresponding to the bounding box annotation are concatenated to obtain a four-channel original input. On the Mapillary dataset, the bounding box annotation is inferred based on the original fine mask annotation in the dataset. That is, for each mask annotation, the bounding box that just completely contains the mask is used as the corresponding bounding box annotation. The pixel value within the bounding box of the binary image is 1, and the rest is 0. The marked border is appropriately enlarged to obtain a cropping area, and the original input is cropped according to the cropping area to obtain the cropped image; The cropped image is input into the segmentation network to output the segmentation mask corresponding to the target object.

3. The weakly supervised video target segmentation method based on border annotation according to claim 2 is characterized in that: The specific method for appropriately enlarging the marked border is as follows: Extend the labeled border by n pixels in the four directions of top, bottom, left, and right.

4. The weakly supervised video object segmentation method based on border annotation according to claim 1 is characterized in that The segmentation network structure used in the pseudo-annotation generation model is the Deeplab-v3+ structure, and the pixel-by-pixel cross entropy function is used as the loss function for model training.

5. The weakly supervised video object segmentation method based on border annotation according to claim 1 is characterized in that The pseudo-annotation generation model is used to generate corresponding pseudo-mask annotations for the video data and the bounding box annotations frame by frame, specifically: Use the video-to-image conversion tool to convert the video data in the Youtube-VOS dataset into image frames; Input each image frame and its corresponding bounding box annotation into the pseudo-annotation generation model to obtain the pseudo mask annotation of each frame; For multiple target objects in the same frame, each object is processed one by one to obtain a pseudo mask annotation for each object when generating a pseudo mask annotation. If two pseudo mask annotations overlap, the overlapping part is considered to belong to the pseudo mask annotation with a smaller area.

6. The weakly supervised video target segmentation method based on border annotation according to claim 5 is characterized in that: Use the ffmpeg tool to convert video data into image frames.

7. The weakly supervised video object segmentation method based on border annotation according to claim 1 is characterized in that The R(T)% parameter gradually increases with the increase in the number of training iterations, that is, during the training process, more and more pseudo mask annotations will be screened out to participate in the training model.

8. A weakly supervised video object segmentation system based on border annotation, applied to the weakly supervised video object segmentation method based on border annotation according to any one of claims 1 to 7, characterized in that: It includes a pseudo-annotation generation model training module, a pseudo-mask annotation generation module, and a segmentation model training module; The pseudo-annotation generation model training module is used to train a pseudo-annotation generation model based on the PReMVOS model on an image segmentation dataset; The pseudo mask annotation generation module is used to generate corresponding pseudo mask annotations for the video data and the bounding box annotations frame by frame using the pseudo annotation generation model; The segmentation model training module is used to use the generated pseudo mask annotations to train the video object segmentation model using a "cooperative teaching" algorithm.

9. A computer-readable storage medium comprising a program, characterized in that: When the program is executed, the weakly supervised video target segmentation method based on border annotation according to any one of claims 1 to 7 is implemented.

Images