Video watermark segmentation method, device, equipment and storage medium

Abstract

The present invention relates to the field of computers, and discloses a video watermark segmentation method, device, equipment and storage medium. The method comprises: pushing a target video, and obtaining a selected video frame image and a selection prompt input in the selected video frame image from the target video; obtaining a target watermark mask image through a trained image segmentation model according to the selected video frame image and the selection prompt; obtaining a target watermark mask image of each video frame in the target video through a trained video target segmentation model according to the target watermark mask image and the target video; and segmenting the target watermark image from each video frame according to the target watermark mask image of each video frame in the target video to obtain a watermark segmentation video. In an embodiment of the present invention, the untrained watermark type can be identified through the selection prompt, and at the same time, the various watermarks in the target video can be distinguished through the video target segmentation model, thereby improving the efficiency of video watermark segmentation.

Description

Technical Field

[0001] The present invention relates to the field of video, and in particular to a video watermark segmentation method, device, equipment and storage medium. Background Art

[0002] In the era of video prevalence, video watermarks have become an important means of protecting intellectual property rights, but in real life, the existence of video watermarks has brought some troubles. From the user's perspective, the watermark pattern on the corner of the video screen will affect the video playback screen and greatly reduce the user's viewing experience; from the video's owner, the video watermark may expire. Therefore, it is of great practical significance to effectively remove video watermarks without affecting the quality of the video.

[0003] The current mainstream video watermark segmentation method mainly identifies the watermarks that appear in all frames of the video, but does not track and distinguish which watermark it is. Moreover, it can only identify trained watermarks and cannot adapt to unknown watermarks, resulting in low efficiency in video watermark segmentation. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problem of low efficiency in video watermark segmentation.

[0005] A first aspect of the present invention provides a video watermark segmentation method, the video watermark segmentation method comprising:

[0006] Pushing a target video, and obtaining a selected video frame image and a selection prompt input in the selected video frame image from the target video, wherein the selection prompt includes one or more of a click prompt and a box selection prompt, and the selection prompt is used to determine the range of the segmented target watermark image on the selected video frame image;

[0007] According to the selected video frame image and the selection prompt, a target watermark mask image is obtained through a trained image segmentation model;

[0008] According to the target watermark mask image and the target video, a target watermark mask image of each video frame in the target video is obtained by using a trained video target segmentation model;

[0009] According to the target watermark mask image of each video frame in the target video, the target watermark image is segmented from each video frame to obtain a watermark segmented video.

[0010] Optionally, in a first implementation of the first aspect of the present invention, obtaining a target watermark mask image by a trained image segmentation model according to the selected video frame image and the selection prompt includes:

[0011] Scaling the selected video frame image according to a preset image size;

[0012] Fill the scaled video frame image with pixels;

[0013] According to the pixel-filled video frame image and the selection prompt, a target watermark mask image is obtained through a trained image segmentation model.

[0014] Optionally, in a second implementation of the first aspect of the present invention, obtaining a target watermark mask image by a trained image segmentation model based on the pixel-filled video frame image and the selection prompt includes:

[0015] Inputting the pixel-filled video frame image into the trained image segmentation model, and obtaining the video frame image feature information through the image encoding processing module of the image segmentation model;

[0016] According to the selection prompt and the video frame image feature information, a target watermark mask image is obtained through the mask prediction module of the image segmentation model.

[0017] Optionally, in a third implementation of the first aspect of the present invention, obtaining a target watermark mask image through a mask prediction module of the image segmentation model according to the selection prompt and the video frame image feature information includes:

[0018] encoding the selection prompt by a prompt encoder;

[0019] The encoded selection prompt and the video frame image feature information are combined and input into the mask prediction module of the image segmentation model, and the target watermark mask image is obtained through decoder prediction.

[0020] Optionally, in a fourth implementation manner of the first aspect of the present invention, obtaining a target watermark mask image for each video frame in the target video through a trained video target segmentation model according to the target watermark mask image and the target video includes:

[0021] Taking out the first video frame of the target video, and performing scaling and pixel filling processing on the first video frame and the target watermark mask image;

[0022] Inputting the processed first video frame and the target watermark mask image into the trained video target segmentation model, and obtaining the watermark image segmentation feature information of the first video frame through the image encoding processing module of the video target segmentation model;

[0023] The watermark image segmentation feature information of the first video frame and all remaining video frames of the target video are respectively input into the trained video target segmentation model, and the target watermark mask image of each video frame is obtained through the mask prediction module of the video target segmentation model.

[0024] Optionally, in a fifth implementation of the first aspect of the present invention, all remaining video frames of the target video are queried from a feature memory, the feature memory including long-term memory, working memory and sensory memory; the target watermark mask image features of each video frame are updated to the sensory memory corresponding to each video frame, and a preset number of video frame features are inserted into the working memory; when the video frames stored in the working memory reaches a preset maximum number of frames, the video frame features in the working memory are integrated into the long-term memory.

[0025] Optionally, in a sixth implementation of the first aspect of the present invention, the video watermark segmentation method further includes:

[0026] Obtain multiple types of watermark templates, and superimpose different types of watermark templates with different data sets through image synthesis to generate non-repetitive watermarked video data;

[0027] Simulating an interactive process, randomly selecting a watermark foreground point or a watermark image frame from the watermarked video data;

[0028] According to the watermark foreground point or the watermark image frame, the target watermark mask image of the watermarked video is iteratively predicted through the initial image segmentation model to obtain a trained image segmentation model.

[0029] The second aspect of the present invention provides a video watermark segmentation device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected through lines; the at least one processor calls the instructions in the memory so that the video watermark segmentation device executes the above-mentioned video watermark segmentation method.

[0030] A third aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned video watermark segmentation method.

[0031] In an embodiment of the present invention, a target video is pushed, and a selected video frame image and a selection prompt entered in the selected video frame image are obtained from the target video, wherein the selection prompt includes one or more of a click prompt and a box selection prompt, and the selection prompt is used to determine the range of the segmented target watermark image on the selected video frame image; based on the selected video frame image and the selection prompt, a target watermark mask image is obtained through a trained image segmentation model; based on the target watermark mask image and the target video, a target watermark mask image of each video frame in the target video is obtained through a trained video target segmentation model; based on the target watermark mask image of each video frame in the target video, the target watermark image is segmented from each video frame to obtain a watermark segmented video. In the present invention, according to the selected video frame image and the selection prompt, a target watermark mask image is obtained through a trained image segmentation model. Then, according to the target watermark mask image and the target video, a target watermark mask image of each video frame in the target video is obtained through a trained video target segmentation model to segment the target watermark image from each video frame. The untrained watermark type can be identified through the selection prompt. At the same time, the various watermarks in the target video can be distinguished through the video target segmentation model, thereby improving the efficiency of video watermark segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of an embodiment of a video watermark segmentation method in an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of an embodiment of a video watermark segmentation device in an embodiment of the present invention;

[0034] Figure 3 It is a schematic diagram of an embodiment of a video watermark segmentation device in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The embodiments of the present invention provide a video watermark segmentation method, apparatus, device and storage medium.

[0036] The embodiments disclosed in the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments disclosed in the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments disclosed in the present invention are only for exemplary purposes and are not intended to limit the scope of protection disclosed in the present invention.

[0037] In the description of the embodiments disclosed in the present invention, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0038] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the video watermark segmentation method in the embodiment of the present invention includes:

[0039] S100, pushing a target video, and acquiring a selected video frame image and a selection prompt input in the selected video frame image from the target video.

[0040] In this embodiment, the video watermark segmentation system pushes the target video, wherein the target video refers to the video to be watermark segmented, and the user selects a video frame image in the target video and enters a selection prompt in the selected video frame image, wherein the selection prompt includes one or more of a click prompt and a box selection prompt, that is, by clicking or boxing the watermark to be processed in the target video. The watermark image to be clicked or boxed in the target video can be one or more.

[0041] S200, obtaining a target watermark mask image through a trained image segmentation model according to the selected video frame image and the selection prompt.

[0042] In this embodiment, the selected video frame image and the selection prompt are input into the trained image segmentation model to obtain the target watermark mask image, wherein the image segmentation model may be SAM (Segment Anything Model), which is an algorithm model for image segmentation based on deep learning. Through the coordinate points given by the user, or prompts such as stretching and box selection, the mask required in the image can be segmented. The SAM model structure consists of an image encoder, a prompt encoder, and a mask decoder, wherein:

[0043] Image encoder: Input an image of (c, h, w), scale the image, scale the long side to 1024, and pad the short side if it is not enough to get an image of (c, 1024, 1024). After the image encoder, get the feature of the image downsampled 16 times, the size is (256, 64, 64).

[0044] prompt encoder: points use position encoding; boxes use a pair of position encodings (upper left and lower right corners)

[0045] Mask decoder: Inserts a learnable token into the prompt embeddings for the output of the docoder.

[0046] (1)prompt toekns+output tokens for self attn,

[0047] (2) Use the obtained token and image embedding for cross attn (token as Q)

[0048] (3) Point-wise MLP updates tokens

[0049] (4) Use the image embedding and the token from (3) to perform cross attenuation (image embedding is used as Q)

[0050] Repeat the above steps twice, then connect attn through the residual, and finally output masks and iou scores.

[0051] S300, according to the target watermark mask image and the target video, a target watermark mask image of each video frame in the target video is obtained through a trained video target segmentation model.

[0052] In this embodiment, a watermark image template is used to synthesize a large amount of video data with different watermarks, a video target segmentation model is trained, and then the target watermark mask image and the target video are input into the trained video target segmentation model to obtain the target watermark mask image of each frame in the target video, wherein the video target segmentation model can be an XMem model, which is a long video target segmentation method based on deep learning, has long-term memory capability, and can effectively process image target segmentation in long videos.

[0053] S400, according to the target watermark mask image of each video frame in the target video, segment the target watermark image from each video frame to obtain a watermark segmentation video.

[0054] In this embodiment, the target watermark image is segmented from each video frame according to the target watermark mask image of each video frame in the target video. For example, the pixel points of the target watermark mask image in each video frame are marked as "1", then the pixel points marked as "1" need to be segmented from each video frame to obtain the watermark segmentation video.

[0055] In an optional implementation of the first aspect of the present invention, according to the selected video frame image and the selection prompt, obtaining the target watermark mask image through the trained image segmentation model includes:

[0056] According to the preset image size, the selected video frame image is scaled; the scaled video frame image is pixel-filled; and according to the pixel-filled video frame image and the selection prompt, a target watermark mask image is obtained through a trained image segmentation model.

[0057] In this embodiment, the selected video frame image (for example, 1920x1080) is first processed to obtain the size required by the pre-trained model (for example, 1024x1024), and the relationship between the long side of the image and the preset value 1024 can be calculated, such as the scaling ratio scale=1024 / 1920=0.53. The selected video frame image is scaled proportionally using the scaling ratio to obtain the scaled image (at this time, the long side of the image meets the requirement, but the short side will be smaller than the required size), and then the scaled image is filled with pixels to the size required by the model; the pixel-filled video frame image and the selection prompt are input into the trained image segmentation model to obtain the target watermark mask image.

[0058] In an optional implementation of the first aspect of the present invention, according to the pixel-filled video frame image and the selection prompt, obtaining the target watermark mask image through the trained image segmentation model includes:

[0059] The pixel-filled video frame image is input into the trained image segmentation model, and the video frame image feature information is obtained through the image encoding processing module of the image segmentation model; according to the selection prompt and the video frame image feature information, the target watermark mask image is obtained through the mask prediction module of the image segmentation model.

[0060] In this embodiment, the pixel-filled video frame image is input into the image encoding processing module of the SAM model to obtain the video frame image feature information; the point information or frame information of the target to be segmented is input by the user, combined with the video frame image feature information, and input into the mask prediction module of the SAM model to output the target watermark mask image; at this time, because the video frame image of the input model has been processed, post-processing recovery is required, first remove the filling, and then scale it back according to the proportion, so as to obtain the original watermark mask image. Figure 1 Sample ratio mask.

[0061] In an optional implementation of the first aspect of the present invention, according to the selection prompt and the video frame image feature information, obtaining the target watermark mask image through the mask prediction module of the image segmentation model includes:

[0062] The selection prompt is encoded through the prompt encoder; the encoded selection prompt and the video frame image feature information are combined and input into the mask prediction module of the image segmentation model, and the target watermark mask image is obtained through the decoder prediction.

[0063] In this embodiment, the SAM model first generates a code for the selected video frame image through an image encoder, that is, the selected video frame image is scaled, and a lightweight encoder is used to convert the user's selection prompt into a prompt code. Then, SAM combines the image code with the prompt code information source respectively, and transmits it to the mask prediction module input to the image segmentation model, and through a lightweight decoder, it is used to predict the segmentation mask.

[0064] In an optional implementation of the first aspect of the present invention, according to the target watermark mask image and the target video, the target watermark mask image of each video frame in the target video is obtained by a trained video target segmentation model, including:

[0065] The first video frame of the target video is taken out, and the first video frame and the target watermark mask image are scaled and pixel-filled; the processed first video frame and the target watermark mask image are input into the trained video target segmentation model, and the watermark image segmentation feature information of the first video frame is obtained through the image encoding processing module of the video target segmentation model; the watermark image segmentation feature information of the first video frame and all the remaining video frames of the target video are respectively input into the trained video target segmentation model, and the target watermark mask image of each video frame is obtained through the mask prediction module of the video target segmentation model.

[0066] In this embodiment, the first video frame image and the target watermark mask image are first preprocessed, mainly to adapt to the model required size, such as 512x512. The processed first video frame image and watermark mask image are input into the image encoding processing module of the Xmem model to obtain the watermark image segmentation feature information of the first video frame. Starting from the second frame of the target video, the watermark image segmentation feature information of the first video frame and the remaining video frames are respectively input into the mask prediction module of the Xmem model to obtain the target watermark mask image of each video frame, and then the target watermark image is segmented from each video frame to obtain the watermark segmentation video. This is actually using the watermark mask image of the first video frame image as a template to segment other video frames, which has a certain range of adaptability compared with previous technologies.

[0067] In an optional implementation of the first aspect of the present invention, all remaining video frames of the target video are queried from a feature memory, the feature memory including long-term memory, working memory and sensory memory; the target watermark mask image features of each video frame are updated to the sensory memory corresponding to each video frame, and a preset number of video frame features are inserted into the working memory; when the video frames stored in the working memory reaches a preset maximum number of frames, the video frame features in the working memory are integrated into the long-term memory.

[0068] In this embodiment, given the first video frame image and the target object mask, XMem tracks the object and generates corresponding masks for subsequent query frames. For each subsequent query frame, memory reading is performed from long-term memory, working memory, and sensory memory respectively. The read features are used to generate segmentation masks. Then, each feature memory is updated at different frequencies. The sensory memory is updated every frame, and the features are inserted into the working memory every r frames. When the working memory reaches the predefined maximum number of frames Tmax, the features in the working memory are integrated into the long-term memory in a highly compact form. When the long-term memory is also full, outdated features are discarded to limit the use of GPU memory. These feature memories work together to provide high-quality features with low GPU memory usage even for very long videos. Among them, the long-term memory is to store a set of compact and representative memory features; the working memory stores high-resolution features in a temporary buffer; the sensory memory focuses on short-term memory and retains low-level information, such as object location.

[0069] In an optional implementation manner of the first aspect of the present invention, the video watermark segmentation method further includes:

[0070] Acquire multiple types of watermark templates, and through image synthesis, superimpose different types of watermark templates with different data sets to generate non-repetitive watermarked video data; simulate the interactive process, randomly select watermark foreground points or watermark image frames from the watermarked video data; according to the watermark foreground points or watermark image frames, iteratively predict the target watermark mask image of the watermarked video through the initial image segmentation model to obtain the trained image segmentation model.

[0071] In this embodiment, common video watermark data sets are collected, and various watermark templates are superimposed on different video data through image synthesis to generate a large amount of non-repetitive watermarked video data. During training, the interactive process is simulated, and foreground points or boxes are randomly selected from the target mask. The points are selected from the gt mask, and the box increases the noise of 10% on the long side, with a maximum of 20 pixels. After the first prompt predicts the mask, the subsequent sampling points are from the area where the predicted mask and the gt mask are different. If the newly generated point is FN (False Negative), it is used as the foreground, and if it is FP (False Positive), it is used as the background. At the same time, the predicted mask is used as a prompt for iteration.

[0072] See also Figure 2 The second aspect of the present invention provides a video watermark segmentation device, the video watermark segmentation device comprising:

[0073] The selection module 100 is used to push the target video and obtain the selected video frame image and the selection prompt input in the selected video frame image from the target video, wherein the selection prompt includes one or more of a click prompt and a box selection prompt, and the selection prompt is used to determine the range of the segmented target watermark image on the selected video frame image;

[0074] The first watermark acquisition module 200 is used to obtain a target watermark mask image according to the selected video frame image and the selection prompt through the trained image segmentation model;

[0075] The second watermark acquisition module 300 is used to obtain the target watermark mask image of each video frame in the target video according to the target watermark mask image and the target video through the trained video target segmentation model;

[0076] The watermark segmentation module 400 is used to segment the target watermark image from each video frame according to the target watermark mask image of each video frame in the target video to obtain a watermark segmentation video.

[0077] In an optional implementation of the second aspect of the present invention, the first watermark acquisition module 200 is also used to scale the selected video frame image according to a preset image size; perform pixel filling on the scaled video frame image; and obtain the target watermark mask image through a trained image segmentation model based on the pixel-filled video frame image and the selection prompt.

[0078] In an optional implementation of the second aspect of the present invention, the first watermark acquisition module 200 is also used to input the pixel-filled video frame image into a trained image segmentation model, and obtain the video frame image feature information through the image encoding processing module of the image segmentation model; according to the selection prompt and the video frame image feature information, the target watermark mask image is obtained through the mask prediction module of the image segmentation model.

[0079] In an optional implementation of the second aspect of the present invention, the first watermark acquisition module 200 is also used to encode the selection prompt through a prompt encoder; the encoded selection prompt and video frame image feature information are combined and input into the mask prediction module of the image segmentation model, and the target watermark mask image is obtained through decoder prediction.

[0080] In an optional implementation of the second aspect of the present invention, the second watermark acquisition module 300 is also used to extract the first video frame of the target video, and scale and pixel-fill the first video frame and the target watermark mask image; input the processed first video frame and the target watermark mask image into the trained video target segmentation model, and obtain the watermark image segmentation feature information of the first video frame through the image encoding processing module of the video target segmentation model; input the watermark image segmentation feature information of the first video frame and all the remaining video frames of the target video into the trained video target segmentation model, and obtain the target watermark mask image of each video frame through the mask prediction module of the video target segmentation model.

[0081] In an optional implementation of the second aspect of the present invention, the second watermark acquisition module 300 is also used to query all remaining video frames of the target video from the feature memory, the feature memory includes long-term memory, working memory and sensory memory; update the target watermark mask image features of each video frame to the sensory memory corresponding to each video frame, and insert a preset number of video frame features into the working memory; when the video frames stored in the working memory reaches a preset maximum number of frames, the video frame features in the working memory are integrated into the long-term memory.

[0082] In an optional implementation of the second aspect of the present invention, the video watermark segmentation device further includes:

[0083] Acquire multiple types of watermark templates, and through image synthesis, superimpose different types of watermark templates with different data sets to generate non-repetitive watermarked video data; simulate the interactive process, randomly select watermark foreground points or watermark image frames from the watermarked video data; according to the watermark foreground points or watermark image frames, iteratively predict the target watermark mask image of the watermarked video through the initial image segmentation model to obtain the trained image segmentation model.

[0084] Figure 31 is a schematic diagram of the structure of a video watermark segmentation device provided by an embodiment of the present invention. The video watermark segmentation device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. Among them, the memory 520 and the storage medium 530 can be short-term storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the video watermark segmentation device 500. Furthermore, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the video watermark segmentation device 500.

[0085] The video watermark segmentation device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Serve, Mac OS X, Unix, Linux, Free BSD, etc. Those skilled in the art will appreciate that Figure 3 The structure of the video watermark segmentation device shown does not constitute a limitation on the video watermark segmentation device, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0086] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the video watermark segmentation method.

[0087] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0088] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination mode.

[0089] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A video watermark segmentation method, characterized in that: The video watermark segmentation method comprises: Pushing a target video, and obtaining a selected video frame image and a selection prompt input in the selected video frame image from the target video, wherein the selection prompt includes one or more of a click prompt and a box selection prompt, and the selection prompt is used to determine the range of the segmented target watermark image on the selected video frame image; According to the selected video frame image and the selection prompt, a target watermark mask image is obtained through a trained image segmentation model; According to the target watermark mask image and the target video, a target watermark mask image of each video frame in the target video is obtained by using a trained video target segmentation model; According to the target watermark mask image of each video frame in the target video, the target watermark image is segmented from each video frame to obtain a watermark segmented video; The step of obtaining the target watermark mask image of each video frame in the target video by training the video target segmentation model according to the target watermark mask image and the target video comprises: Taking out the first video frame of the target video, and performing scaling and pixel filling processing on the first video frame and the target watermark mask image; Inputting the processed first video frame and the target watermark mask image into the trained video target segmentation model, and obtaining the watermark image segmentation feature information of the first video frame through the image encoding processing module of the video target segmentation model; Input the watermark image segmentation feature information of the first video frame and all the remaining video frames of the target video into the trained video target segmentation model respectively, and obtain the target watermark mask image of each video frame through the mask prediction module of the video target segmentation model; The video watermark segmentation method further comprises: obtaining multiple types of watermark templates, and superimposing different types of watermark templates with different data sets through image synthesis to generate non-repetitive watermarked video data; Simulating an interactive process, randomly selecting a watermark foreground point or a watermark image frame from the watermarked video data; According to the watermark foreground point or the watermark image frame, the target watermark mask image of the watermarked video is iteratively predicted by the initial image segmentation model to obtain a trained image segmentation model; During the training, the simulated interaction process randomly selects foreground points or image frames from the target mask. The foreground points are selected from the real target mask annotated by the gt mask. The image frame increases the noise of 10% on the long side, with a maximum of 20 pixels. After the first prompt word predicts the mask, the subsequent sampling points are from the area where the predicted mask and the gt mask are different. If the newly sampled point is an FN false negative example, it is used as the foreground. If it is an FP false positive example, it is used as the background. At the same time, the predicted mask is used as the prompt iteration.

2. The video watermark segmentation method according to claim 1, characterized in that: The step of obtaining a target watermark mask image according to the selected video frame image and the selection prompt by using a trained image segmentation model comprises: Scaling the selected video frame image according to a preset image size; Fill the scaled video frame image with pixels; According to the pixel-filled video frame image and the selection prompt, a target watermark mask image is obtained through a trained image segmentation model.

3. The video watermark segmentation method according to claim 2, characterized in that: The step of obtaining a target watermark mask image by training an image segmentation model based on the pixel-filled video frame image and the selection prompt includes: Inputting the pixel-filled video frame image into the trained image segmentation model, and obtaining the video frame image feature information through the image encoding processing module of the image segmentation model; According to the selection prompt and the video frame image feature information, a target watermark mask image is obtained through the mask prediction module of the image segmentation model.

4. The video watermark segmentation method according to claim 3, characterized in that: The step of obtaining a target watermark mask image according to the selection prompt and the video frame image feature information through a mask prediction module of the image segmentation model comprises: encoding the selection prompt by a prompt encoder; The encoded selection prompt and the video frame image feature information are combined and input into the mask prediction module of the image segmentation model, and the target watermark mask image is obtained through decoder prediction.

5. The video watermark segmentation method according to claim 1, characterized in that: Query all remaining video frames of the target video from the feature memory, the feature memory includes long-term memory, working memory and sensory memory; update the target watermark mask image features of each video frame to the sensory memory corresponding to each video frame, and insert a preset number of video frame features into the working memory; when the video frames stored in the working memory reaches a preset maximum number of frames, integrate the video frame features in the working memory into the long-term memory.

6. A video watermark segmentation device, characterized in that: The video watermark segmentation device comprises: A selection module is used to push a target video, and obtain a selected video frame image from the target video, and a selection prompt input in the selected video frame image, wherein the selection prompt includes one or more of a click prompt and a box selection prompt, and the selection prompt is used to determine the range of the segmented target watermark image on the selected video frame image; A first watermark acquisition module, configured to obtain a target watermark mask image according to the selected video frame image and the selection prompt through a trained image segmentation model; A second watermark acquisition module is used to obtain a target watermark mask image of each video frame in the target video according to the target watermark mask image and the target video through a trained video target segmentation model; A watermark segmentation module is used to segment the target watermark image from each video frame according to the target watermark mask image of each video frame in the target video to obtain a watermark segmentation video; The second watermark acquisition module is also used to take out the first video frame of the target video, and perform scaling and pixel filling processing on the first video frame and the target watermark mask image; Inputting the processed first video frame and the target watermark mask image into the trained video target segmentation model, and obtaining the watermark image segmentation feature information of the first video frame through the image encoding processing module of the video target segmentation model; Input the watermark image segmentation feature information of the first video frame and all the remaining video frames of the target video into the trained video target segmentation model respectively, and obtain the target watermark mask image of each video frame through the mask prediction module of the video target segmentation model; The video watermark segmentation device also includes: obtaining multiple types of watermark templates, and superimposing different types of watermark templates with different data sets through image synthesis to generate non-repetitive watermarked video data; Simulating an interactive process, randomly selecting a watermark foreground point or a watermark image frame from the watermarked video data; According to the watermark foreground point or the watermark image frame, the target watermark mask image of the watermarked video is iteratively predicted by the initial image segmentation model to obtain a trained image segmentation model; During the training, the simulated interaction process randomly selects foreground points or image frames from the target mask. The foreground points are selected from the real target mask annotated by the gt mask. The image frame increases the noise of 10% on the long side, with a maximum of 20 pixels. After the first prompt word predicts the mask, the subsequent sampling points are from the area where the predicted mask and the gt mask are different. If the newly sampled point is an FN false negative example, it is used as the foreground. If it is an FP false positive example, it is used as the background. At the same time, the predicted mask is used as the prompt iteration.

7. A video watermark segmentation device, characterized in that: The video watermark segmentation device comprises: a memory and at least one processor, the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor calls the instructions in the memory to enable the video watermark segmentation device to perform the video watermark segmentation method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video watermark segmentation method according to any one of claims 1 to 5 is implemented.

Images