Training method and device of video segmentation model, segmentation method and device, equipment and medium

Abstract

The embodiment of the invention provides a training method and device of a video segmentation model, a segmentation method and device, equipment and a medium. The method comprises the following steps: acquiring a sample picture and a sample video; adjusting a pixel position corresponding to a target object in the sample picture to obtain an analog video, and determining a training sample set according to the sample video and the analog video; inputting the video frames in the training sample set into a to-be-trained video segmentation model, and determining a prediction mask of a target object in the video frames through the video segmentation model; and determining a loss function value according to the prediction mask and the target mask of the video frame, and correcting model parameters of the video segmentation model according to the loss function value. According to the embodiment of the invention, the consistency and stability of the video segmentation result are improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to computer technology, and more particularly to a training method, segmentation method, apparatus, device and medium for a video segmentation model. Background Art

[0002] Video segmentation is a technique for segmenting target objects from a video, where the target objects may include people or animals.

[0003] At present, video segmentation processes videos as independent images. If the target object in the video is moving, some areas of the target object may be identified as the background, resulting in unstable segmentation results for the target object. In particular, when the target object in the video only contains a bust, the problem of unstable segmentation is more obvious. For example, if only the upper body of a person appears in the video, some video frames may correctly segment the person, while other video frames may mistakenly identify the person's belly as the background, resulting in the person in the video sometimes having a belly and sometimes not having a belly. Summary of the invention

[0004] The embodiments of the present disclosure provide a video segmentation model training method, segmentation method, device, equipment and medium, which can improve the consistency and stability of video segmentation results.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for training a video segmentation model, comprising:

[0006] Acquire sample pictures and sample videos, wherein the sample pictures and sample videos include a bust of the target object;

[0007] Adjusting the pixel position corresponding to the target object in the sample image to obtain a simulated video, and determining a training sample set according to the sample video and the simulated video;

[0008] Inputting the video frames in the training sample set into a video segmentation model to be trained, and determining a predicted mask of a target object in the video frame by using the video segmentation model;

[0009] A loss function value is determined according to a predicted mask and a target mask of the video frame, and a model parameter of the video segmentation model is corrected according to the loss function value, wherein the target mask represents the target object in the video frame.

[0010] In a second aspect, the present disclosure also provides a video segmentation method, including:

[0011] Acquire a video to be segmented, wherein a video frame of the video to be segmented includes a bust of a target object;

[0012] Inputting the video to be segmented into a video segmentation model, and outputting an object mask corresponding to the target object in the video to be segmented through the video segmentation model, wherein the video segmentation model is trained using the training method of the video segmentation model described in any embodiment of the present disclosure;

[0013] The target object in the video to be segmented is segmented according to the object mask corresponding to the target object in the video to be segmented.

[0014] In a third aspect, the present disclosure also provides a video segmentation model training device, the device comprising:

[0015] A first video acquisition module, used to acquire sample pictures and sample videos, wherein the sample pictures and sample videos include a bust of a target object;

[0016] A sample determination module, used to adjust the pixel position corresponding to the target object in the sample image to obtain a simulated video, and determine a training sample set according to the sample video and the simulated video;

[0017] A first mask determination module, configured to input the video frame in the training sample set into a video segmentation model to be trained, and determine a predicted mask of a target object in the video frame by using the video segmentation model;

[0018] A parameter correction module is used to determine a loss function value according to a predicted mask and a target mask of the video frame, and to correct model parameters of the video segmentation model according to the loss function value, wherein the target mask represents the target object in the video frame.

[0019] In a fourth aspect, an embodiment of the present disclosure further provides a video segmentation device, the device comprising:

[0020] A second video acquisition module is used to acquire a video to be segmented, wherein the video frame of the video to be segmented includes a bust of the target object;

[0021] A second mask determination module is used to input the video to be segmented into a video segmentation model, and output an object mask corresponding to the target object in the video to be segmented through the video segmentation model, wherein the video segmentation model is trained using the training method of the video segmentation model described in any embodiment of the present disclosure;

[0022] The object segmentation module is used to segment the target object in the video to be segmented according to the object mask corresponding to the target object in the video to be segmented.

[0023] In a fifth aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0024] one or more processors;

[0025] a storage device for storing one or more programs,

[0026] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any embodiment of the present disclosure.

[0027] In a sixth aspect, an embodiment of the present disclosure further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the method as described in any embodiment of the present disclosure.

[0028] The disclosed embodiments provide a training method, a segmentation method, an apparatus, a device and a medium for a video segmentation model, which can generate a simulated video by adjusting the position of the pixels corresponding to the target object in the sample image, thereby increasing the number of half-body training samples. A training sample set is determined based on the sample video and the simulated video, and the video frame in the training sample set is input into the video segmentation model to be trained, and the predicted mask of the target object in the video frame output by the video segmentation model is obtained. The loss function value is determined based on the predicted mask and the target mask of the video frame, and the video segmentation model is trained based on the loss function value, so that the video segmentation model can accurately learn the characteristics of the target object in the half-body scene, thereby solving the problem of unstable video segmentation and achieving the effect of improving the consistency and stability of the video segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0030] Figure 1 A flowchart of a video segmentation model training method provided by an embodiment of the present disclosure;

[0031] Figure 2 A schematic diagram of the structure of a video segmentation model provided by an embodiment of the present disclosure;

[0032] Figure 3 A flowchart of another video segmentation model training method provided by an embodiment of the present disclosure;

[0033] Figure 4 A flowchart of another video segmentation model training method provided by an embodiment of the present disclosure;

[0034] Figure 5 A schematic diagram of a flow chart of a video segmentation method provided by an embodiment of the present disclosure;

[0035] Figure 6 A schematic diagram of the structure of a video segmentation model training device provided by an embodiment of the present disclosure;

[0036] Figure 7 A schematic diagram of the structure of a video segmentation device provided by an embodiment of the present disclosure;

[0037] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0039] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0040] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0041] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0042] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0043] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0044] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0045] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0046] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0047] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0048] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0049] Figure 1 The present invention is a flowchart of a video segmentation model training method provided by an embodiment of the present invention. The present invention is applicable to a half-body segmentation scene of a target object in a video, for example, a portrait segmentation in a video. The method can be executed by a video segmentation model training device, which can be implemented in the form of software and / or hardware, and optionally, by an electronic device, which can be a mobile terminal, a PC, or a server.

[0050] like Figure 1 As shown, the method includes:

[0051] S110: Obtain sample pictures and sample videos, wherein the sample pictures and sample videos include a bust of the target object.

[0052] The sample images may be images used to form a training sample set. The sample videos may be videos used to form a training sample set. The target objects may be foreground objects in the sample images and sample videos. For example, the target objects may include people or animals.

[0053] In the disclosed embodiment, data cleaning can be performed on the pictures and videos in the image set to remove low-resolution data, as well as data in which the area of ​​the bust in the picture or video frame accounts for less than a set ratio threshold, etc., to obtain sample pictures and sample videos.

[0054] Sample images and sample videos can be generated by image generation technology. Specifically, the user inputs a picture description text, and a sample image is generated by a text-generated image based on the picture description text. The user can also input a video description text, and a sample video is generated by a text-generated video based on the video description text.

[0055] In the disclosed embodiment, in order to increase the proportion of samples in which the target object is a bust of the target object in the sample pictures and sample videos, object recognition is performed on the pictures or videos during the process of obtaining the sample pictures and sample videos. If the picture or video includes a bust of the target object and the area occupied by the bust in the picture or video frame accounts for a proportion greater than a set ratio threshold, the picture or video is used as a sample picture or sample video.

[0056] S120, adjusting the pixel position corresponding to the target object in the sample image to obtain a simulated video, and determining a training sample set according to the sample video and the simulated video.

[0057] Among them, the simulated video can represent a sequence of pictures obtained by arranging the sample pictures after the pixel positions are adjusted in the order of displacement operations. Based on the sample pictures, target pictures corresponding to multiple sample pictures can be produced by moving the pixels of the target object as a whole, and the sample pictures and the target pictures are combined in the order of displacement operations to obtain a simulated video. By generating videos from pictures, the number of videos in the training sample set can be increased and the difficulty of obtaining sample videos can be reduced. Among them, the target picture represents a new picture generated after the pixel points of the target object in the sample picture are displaced. The difference between the target picture and the corresponding sample picture is that the position of the target object is different. The displacement operation order represents the order of movement of the pixel position in the process of adjusting the pixel position of the sample picture.

[0058] Exemplarily, the pixel positions corresponding to the target object in the sample image are adjusted to obtain a simulated video, including: performing N displacement operations on the pixels corresponding to the target object in the sample image along the target direction to obtain N target images; and combining the sample image and the N target images according to the displacement operation sequence to obtain a simulated video.

[0059] For example, the pixels of the target object in the sample image are shifted to the left N times continuously, and each shift is x distance, to obtain N target images. Taking the sample image as the starting point, the N target images are combined with the sample image in the order of displacement operations to obtain a simulated video, which shows the effect of the target object shifting to the left.

[0060] Optionally, the pixels of the target object in the sample image may be continuously translated N times in the clockwise direction to obtain N target images, and the sample image and the N target images may be combined to obtain a simulated video showing the effect of the target object rotating clockwise.

[0061] In the disclosed embodiment, the training sample set represents a set of training samples, and the training samples may be video frames of sample videos or simulated videos. Exemplarily, determining the training sample set based on the sample videos and simulated videos includes: for the video frames in the sample videos or simulated videos, determining the timing information of the video frames and the target mask of the target object. The target mask is used to segment the target object from the video frame. The timing information is added to the feature tensor corresponding to the video frame to obtain the target feature tensor corresponding to the video frame, wherein the target feature tensor is used to characterize the video frame. The training sample set is determined based on the target feature tensor and the target mask.

[0062] Among them, the timing information represents the position of the video frame in the video segment to which it belongs. For example, the timing information of the video frame represents the frame number of the video frame in the video segment to which the video frame belongs. The timing information can be used to represent the timing relationship between video frames in the video segment. In the field of computer vision, feature tensors are usually used to represent image data. For example, image data can be represented in the bchw format, where b represents the batch dimension (batch), which is used to represent the number of samples input at one time. During the training process, inputting multiple samples at one time can improve the computational efficiency. c represents the channel dimension (channel), which is used to represent the number of channels of the input data or the number of feature maps. In RGB images, there are three channels (red, green, and blue). For convolutional neural networks (CNN), it usually represents the number of output feature maps of the convolution layer. h represents the height dimension (height), which is used to represent the height of the image or feature map. w represents the width dimension (width), which is used to represent the width of the image or feature map. The target feature tensor representation adds the timing information t to the dimension of bchw to represent the image data in the bchwt format.

[0063] In the disclosed embodiment, the target mask is used to characterize the target object in the video frame. The target mask of the target object can be determined based on the annotation information of the target object. For example, the pixels corresponding to the target object in the video frame can be annotated as 1, and the other pixels can be annotated as 0, thereby obtaining the target mask of the target object. The target mask in the video frame can be annotated by the model.

[0064] Optionally, at least two pre-trained recognition models are used to identify the target object in the video frame of the sample video or the simulated video to obtain at least two candidate masks of the target object in the video frame. Accurate sample annotation information, i.e., the target mask, is determined according to the overlap of the at least two candidate masks.

[0065] After labeling the video frames of the sample video and the simulation video, the target mask of the target object in the video frame of the sample video and the target mask of the target object in the video frame of the simulation video are obtained. A training sample set is formed according to the video frames and the target masks of the sample video and the simulation video. The video frame is represented in the format of a target feature tensor as the input of the video segmentation model to be trained, and the target mask corresponding to the video frame is used as supervision data to train the video segmentation model. The video segmentation model can be a deep learning model for segmenting target objects in a video.

[0066] S130: Input the video frames in the training sample set into a video segmentation model to be trained, and determine a predicted mask of the target object in the video frame by using the video segmentation model.

[0067] The predicted mask represents the mask of the target object in the video frame in the training sample set determined by the video segmentation model.

[0068] Exemplarily, according to a target sampling ratio, video frames of a sample video or a simulated video are input into a video segmentation model to be trained, wherein the target sampling ratio represents the ratio of the number of times the sample video is used to train the video segmentation model to the number of times the simulated video is used to train the video segmentation model during at least two model iterations; and the predicted mask of the target object in the video frame output by the video segmentation model is obtained.

[0069] Since analog videos may have time discontinuity problems, when selecting training samples, sample videos and analog videos can be used for mixed training to balance the number of samples and the continuity of the time information learned by the model. In order to reduce the impact of the time discontinuity problem of analog videos on model training, a model iteration process using analog videos can be interspersed after using sample videos to train the model at least twice. For example, you can first randomly select a sample video from the training sample set to train the video segmentation model. After completing this model iteration, you can randomly select a sample video from the training sample set to train the video segmentation model again. After completing this model iteration, continue to randomly select a sample video from the training sample set to train the video segmentation model. After executing n cycles, randomly select a simulated video from the training sample set to train the video segmentation model, and then continue to iterate n times using sample videos to train the video segmentation model.

[0070] In the disclosed embodiment, a video frame in a sample video in a training sample set or a video frame in a simulated video is input into a video segmentation model to be trained, and a predicted mask of a target object in the video frame output by the video segmentation model is obtained.

[0071] Figure 2 FIG. 1 is a schematic diagram of a video segmentation model provided by an embodiment of the present disclosure. Figure 2 As shown, the video segmentation model includes an encoder 210, a decoder 220 and a depth-guided filter module 230. The encoder 210 includes a downsampling layer 211, through which the input high-definition video frame is processed into a standard-definition video frame. Then, a plurality of serially connected encoding modules 212 are used to extract multi-scale features of the standard-definition video frame to obtain hidden layer features. The decoder 220 includes at least one cyclic decoding module 221 and an output module 222, and at least one cyclic decoding module 221 is serially connected in sequence. The cyclic decoding module 221 includes a convolutional layer, a normalization layer, an activation layer, a convolutional gated cyclic unit and an upsampling layer, etc., which are connected in sequence. The cyclic decoding module 221 can learn the feature information to be retained and the feature information to be forgotten in a continuous video stream, and store the hidden layer features in a memory pool for processing the next video frame image. The convolutional gated cyclic unit can be a ConvGRU, and the time information is aggregated by the ConvGRU. Connecting the ConvGRU at the front end of the upsampling layer of each cyclic decoding module 221 can ensure that the prediction mask corresponding to different video frames has stability. The target feature tensor corresponding to the current video frame is processed by the convolution layer, the normalization layer and the activation layer, and the processed target feature tensor x_t is input into the segmentation operator. The feature channels of the target feature tensor x_t are halved by the segmentation operator and then input into the convolution gated loop unit. The target feature tensor of half the channels of the hidden layer feature x_t-1 corresponding to the previous video frame is obtained by another segmentation operator, and the target feature tensor of half the channels of x_t-1 is input into the convolution gated loop unit. The hidden layer features of the current video frame are determined by the convolution gated loop unit according to the hidden layer features corresponding to the cached previous video frame, and the ConvGRU output result of the current video frame is determined according to the target feature tensor inputted by the current video frame. The output result of the ConvGRU is fused with the target feature tensor of the other half channels of x_t to obtain the output result of the cyclic decoding module 221 as the input of the next cyclic decoding module 221. Among them, the convolution kernel of the ConvGRU is a 1*1 convolution kernel.

[0072] In the decoder, except for the first loop decoding module 221, the input of each loop decoding module 221 includes the features of the standard definition image after passing through the average pooling layer. For the first loop decoding module 221, the input includes the output features of the encoding module 212. For the second loop decoding module 221, the input includes the features of the standard definition image after passing through the average pooling layer, the output features of the encoding module 212, and the output result of the first loop decoding module 221. The output module 222 includes a convolution layer, a normalization layer, and an activation layer.

[0073] The depth-guided filter module 230 generates a high-resolution prediction mask based on the low-resolution mask, the target mask, and the high-resolution video frame.

[0074] S140, determining a loss function value according to a predicted mask and a target mask of the video frame, and correcting model parameters of the video segmentation model according to the loss function value, wherein the target mask represents the target object in the video frame.

[0075] Since the loss function value is used to quantify the gap between the model prediction value and the true value, in order to optimize the video segmentation model, the model parameters can be adjusted according to the loss function value so that the predicted mask is closer to the target mask. According to the predicted loss value calculated based on the predicted mask and the target mask corresponding to the video frame, the loss function value is determined according to the predicted loss value, and the model parameters of the video segmentation model are updated according to the loss function value to reduce the loss function value. When the loss function value meets the preset model training end condition, the training is terminated to obtain the video segmentation model. Alternatively, when the number of model iterations reaches the preset number threshold, the training is terminated to obtain the video segmentation model.

[0076] The technical solution of the disclosed embodiment generates a simulated video by adjusting the position of the pixels corresponding to the target object in the sample image, so as to increase the number of half-body training samples. A training sample set is determined based on the sample video and the simulated video, and the video frame in the training sample set is input into the video segmentation model to be trained, and the predicted mask of the target object in the video frame output by the video segmentation model is obtained. The loss function value is determined based on the predicted mask and the target mask of the video frame, and the video segmentation model is trained based on the loss function value, so that the video segmentation model can accurately learn the characteristics of the target object in the half-body scene, thereby solving the problem of unstable video segmentation and achieving the effect of improving the consistency and stability of the video segmentation results.

[0077] Figure 3 This is a flow chart of another video segmentation model training method provided by the embodiment of the present disclosure. Based on the above embodiments, the embodiment of the present disclosure further defines the sample annotation method. Figure 3 As shown, the method includes:

[0078] S310: Obtain sample pictures and sample videos, wherein the sample pictures and sample videos include a bust of the target object.

[0079] S320: Adjust the pixel position corresponding to the target object in the sample image to obtain a simulated video.

[0080] S330: Segment the sample video and the simulation video to obtain video clips.

[0081] The video clips include sample video clips and simulation video clips.

[0082] Exemplarily, according to a preset video segmentation rule, the sample video is segmented into at least two sample video segments. In addition, according to a preset video segmentation rule, the simulated video is segmented into at least two simulated video segments. The video segmentation rule can be set according to actual application requirements. The video segmentation rule is used to specify the number of video frames in each video segment. For example, every 15 frames in the sample video can be used as a video segment.

[0083] S340: For a video frame in the video segment, determine timing information of the video frame according to a position of the video frame in the video segment.

[0084] Since the video frames in the video clip are arranged in the order of timestamps, the timing information of the video frames can be determined according to the position of the video in the video clip. For example, the timing information of the third frame of video in the video clip is the third video frame in the video clip.

[0085] Video frames of different video clips usually have low similarity, so the timing information of the video frames can be determined based on the positions of the video frames in the same video clip.

[0086] S350: Perform target object recognition on the video frame using at least two pre-trained recognition models to obtain at least two candidate masks of the target object in the video frame.

[0087] The recognition model is used to recognize the target object in the video frame. The video frame is input into at least two pre-trained recognition models respectively to obtain the candidate mask of the target object in the video frame output by the recognition model. Since at least two recognition models are used to recognize the target object, at least two candidate masks can be obtained.

[0088] S360: Determine an overlap degree according to the at least two candidate masks, and determine a target mask of the target object in the video frame according to the overlap degree and the at least two candidate masks.

[0089] The overlap degree is used to characterize the similarity between at least two candidate masks. If most of the pixels in at least two candidate masks are the same, it is determined that the similarity between at least two candidate masks is high. Optionally, the overlap degree is determined based on the intersection and union of the at least two candidate masks; if the overlap degree is greater than a set overlap threshold, the target mask of the target object in the video frame is determined based on any candidate mask of the at least two candidate masks.

[0090] Specifically, the intersection of pixels in at least two candidate masks is calculated, the union of pixels in at least two candidate masks is calculated, and the quotient of the intersection and the union is used to represent the overlap of at least two candidate masks. If the overlap is greater than the set overlap threshold, it is determined that at least two candidate masks are similar, and any one of the at least two candidate masks can be selected as the target mask of the target object in the video frame. By annotating the video frame with at least two recognition models, the effect of mutual verification is achieved, thereby improving the accuracy of annotation.

[0091] S370. Add the timing information to the feature tensor corresponding to the video frame to obtain a target feature tensor corresponding to the video frame, wherein the target feature tensor is used to characterize the video frame.

[0092] S380: Determine a training sample set according to the target feature tensor and the target mask.

[0093] S390: Input the video frames in the training sample set into the video segmentation model to be trained, and determine the predicted mask of the target object in the video frame by using the video segmentation model.

[0094] S3100, determining a loss function value according to a predicted mask and a target mask of the video frame, and correcting model parameters of the video segmentation model according to the loss function value, wherein the target mask represents the target object in the video frame.

[0095] The technical solution of the disclosed embodiment determines the timing information of the video frame by the position of the video frame in the corresponding video segment, and represents the video frame in combination with the timing information dimension, batch dimension, channel dimension, height dimension and width dimension, so as to facilitate the video segmentation model to learn the timing association between video frames. The target object is labeled by at least two pre-trained recognition models to obtain at least two candidate masks, and the target mask of the target object is determined according to the overlap of at least two candidate masks, so as to realize mutual verification of at least two model labeling results, improve the accuracy of model labeling, and obtain more accurate labeling samples.

[0096] Figure 4This is a flow chart of another video segmentation model training method provided by the embodiment of the present disclosure. Based on the above embodiments, the embodiment of the present disclosure specifically defines the model parameters of the video segmentation model to be corrected according to the loss function value. Figure 4 As shown, the method includes:

[0097] S410: Obtain sample images and sample videos, wherein the sample images and sample videos include a bust of the target object.

[0098] S420: Adjust the pixel position corresponding to the target object in the sample image to obtain a simulated video, and determine a training sample set according to the sample video and the simulated video.

[0099] S430: Input the video frames in the training sample set into a video segmentation model to be trained, and determine the predicted mask of the target object in the video frame by using the video segmentation model.

[0100] S440, determining a loss function value according to the predicted mask, the target mask of the video frame, and the target mask of the previous video frame, and correcting the model parameters of the video segmentation model according to the loss function value.

[0101] Exemplarily, a prediction deviation is determined based on a prediction mask and a target mask of the video frame, and a first loss function value is determined based on the prediction deviation. A target deviation is determined based on a target mask of the video frame and a target mask of a previous video frame, and a second loss function value is determined based on the target deviation. Model parameters of the video segmentation model are corrected based on the first loss function value, wherein the decoder of the video segmentation model includes at least one convolutional gated recurrent unit, the convolutional gated recurrent unit is used to determine the hidden layer features of the video frame based on the hidden layer features corresponding to the previous video frame, and the model parameters include parameters of an update gate in the convolutional gated recurrent unit. The parameters of the update gate are corrected based on the second loss function value.

[0102] In order to reduce the loss of model performance caused by the 1*1 convolution kernel used in the convolution gated recurrent unit, the parameter update of the update gate in the convolution gated recurrent unit is supervised by the difference of the target mask between two adjacent video frames to enhance the feature update capability of the update gate. The update gate is used to determine how much of the hidden layer features of the previous video frame to retain in the hidden layer features of the current video frame.

[0103] The output value of the update gate in the convolutional gated recurrent unit is used as a weight coefficient to calculate how many hidden layer features of the previous video frame are retained in the hidden layer features of the current video frame. If the output value of the update gate is larger, more hidden layer features of the previous video frame need to be retained in the hidden layer features of the current video frame. Therefore, the training of the update gate can be supervised based on the difference in the target mask of adjacent video frames.

[0104] The technical solution of the disclosed embodiment determines the first loss function value according to the predicted deviation between the predicted mask of the video frame and the target mask. The model parameters of the video segmentation model are updated according to the first loss function value. The model parameters include the weight coefficient and bias term of the update gate in the convolution gated recurrent unit. On the other hand, the second loss function value is determined according to the difference between the target mask of the video frame and the previous video frame. By updating the weight coefficient and bias term of the update gate according to the second loss function value, the role of the update gate can be further enhanced, and the loss of model performance caused by the reduction of the convolution kernel can be reduced, so as to achieve the effect of balancing the model calculation amount and model performance.

[0105] Figure 5 The present invention is a flowchart of a video segmentation method provided by an embodiment of the present invention. The present invention is applicable to a half-body segmentation scene of a target object in a video, for example, a portrait segmentation in a video. The method can be executed by a video segmentation device, which can be implemented in the form of software and / or hardware, and optionally, by an electronic device, which can be a mobile terminal, a PC or a server.

[0106] like Figure 5 As shown, the method includes:

[0107] S510: Acquire a video to be segmented, wherein a video frame of the video to be segmented includes a bust of a target object.

[0108] The video to be segmented represents the video to be segmented for the target object. The video to be segmented can be a video in a live broadcast, real-time communication, or interactive entertainment scenario. The bust represents the upper and lower body or left and right body of the target object in the video to be segmented. Since only the bust is shown in the video frame of the video to be segmented, it is difficult for the portrait segmentation algorithm to accurately distinguish between the background and the portrait area, and there is a situation where the portrait segmentation results between video frames are unstable.

[0109] S520: Input the video to be segmented into a video segmentation model, and output an object mask corresponding to the target object in the video to be segmented through the video segmentation model.

[0110] The video segmentation model is trained using the video segmentation model training method described in any embodiment of the present disclosure. The object mask is a mask of the target object determined by the video segmentation model. The object mask is used to segment the target object from the video frame.

[0111] Exemplarily, before the video frames of the video to be segmented are input into the video segmentation model, the video frames are subjected to image preprocessing to achieve standardization according to the image channels, which can accelerate the convergence speed of the model. The preprocessed video frames are input into the video segmentation model, and after the encoder of the video segmentation model performs feature extraction on the preprocessed video frames, the features of the video frames are input into the decoder, and the features of the video frames are decoded by the cyclic decoding module in the decoder, and the hidden layer features of the video frame are determined by the convolution gated cyclic unit according to the hidden layer features corresponding to the previous video frame. The predicted mask of the video frame is determined by the depth-guided filter module according to the decoding result of the video frame and the hidden layer features.

[0112] S530: Segment the target object in the video to be segmented according to the object mask corresponding to the target object in the video to be segmented.

[0113] Exemplarily, for each video frame in the video to be segmented, target object segmentation is performed on the current video frame according to the object mask corresponding to the target object.

[0114] The technical solution of the disclosed embodiment ensures the stability and consistency of the segmentation results by assigning the video to be segmented to a video segmentation model, outputting the object mask corresponding to the target object in the video to be segmented through the video segmentation model, and segmenting the target object in the video to be segmented according to the object mask.

[0115] Figure 6 A schematic diagram of the structure of a video segmentation model training device provided in an embodiment of the present disclosure. The device can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server.

[0116] like Figure 6 As shown, the apparatus includes: a first video acquisition module 610 , a sample determination module 620 , a first mask determination module 630 and a parameter correction module 640 .

[0117] A first video acquisition module 610 is used to acquire sample pictures and sample videos, wherein the sample pictures and sample videos include a bust of a target object;

[0118] A sample determination module 620 is used to adjust the pixel position corresponding to the target object in the sample image to obtain a simulated video, and determine a training sample set according to the sample video and the simulated video;

[0119] A first mask determination module 630, configured to input the video frame in the training sample set into a video segmentation model to be trained, and determine a predicted mask of a target object in the video frame by using the video segmentation model;

[0120] The parameter correction module 640 is used to determine a loss function value according to a predicted mask and a target mask of the video frame, and to correct model parameters of the video segmentation model according to the loss function value, wherein the target mask represents the target object in the video frame.

[0121] Optionally, the sample determination module 620 is specifically configured to:

[0122] Performing N displacement operations on pixels corresponding to the target object in the sample image along the target direction to obtain N target images;

[0123] The sample image and N target images are combined according to the shift operation sequence to obtain a simulated video.

[0124] Optionally, the sample determination module 620 is further configured to:

[0125] For a video frame in the sample video or the simulated video, determining timing information of the video frame and a target mask of a target object;

[0126] Adding the timing information to a feature tensor corresponding to the video frame to obtain a target feature tensor corresponding to the video frame, wherein the target feature tensor is used to characterize the video frame;

[0127] A training sample set is determined according to the target feature tensor and the target mask.

[0128] Furthermore, for a video frame in the sample video or the simulation video, determining the timing information of the video frame and the target mask of the target object includes:

[0129] Segmenting the sample video and the simulation video to obtain video clips;

[0130] For a video frame in the video segment, determining timing information of the video frame according to a position of the video frame in the video segment;

[0131] Performing target object recognition on the video frame using at least two pre-trained recognition models to obtain at least two candidate masks of the target object in the video frame;

[0132] The overlap degree is determined according to the at least two candidate masks, and the target mask of the target object in the video frame is determined according to the overlap degree and the at least two candidate masks.

[0133] Further, determining the overlap degree according to the at least two candidate masks, and determining the target mask of the target object in the video frame according to the overlap degree and the at least two candidate masks, comprises:

[0134] Determining a degree of overlap based on the intersection and union of the at least two candidate masks;

[0135] If the overlap degree is greater than a set overlap threshold, a target mask of the target object in the video frame is determined according to any candidate mask of the at least two candidate masks.

[0136] Optionally, the first mask determination module 630 is specifically configured to:

[0137] Inputting video frames of the sample video or the simulated video into the video segmentation model to be trained according to a target sampling ratio, wherein the target sampling ratio represents a ratio of the number of times the sample video is used to train the video segmentation model to the number of times the simulated video is used to train the video segmentation model during at least two model iterations;

[0138] Obtain a predicted mask of a target object in the video frame output by the video segmentation model.

[0139] Optionally, the parameter correction module 640 is specifically used to:

[0140] A loss function value is determined according to the predicted mask, the target mask of the video frame and the target mask of the previous video frame, and a model parameter of the video segmentation model is corrected according to the loss function value.

[0141] Furthermore, determining a loss function value according to the predicted mask, the target mask and the target mask of the previous video frame, and correcting the model parameters of the video segmentation model according to the loss function value includes:

[0142] Determine a prediction deviation according to a prediction mask and a target mask of the video frame, and determine a first loss function value according to the prediction deviation;

[0143] Determine a target deviation according to a target mask of the video frame and a target mask of a previous video frame, and determine a second loss function value according to the target deviation;

[0144] Correcting model parameters of the video segmentation model according to the first loss function value, wherein the decoder of the video segmentation model includes at least one convolutional gated loop unit, the convolutional gated loop unit is used to determine the hidden layer features of the video frame according to the hidden layer features corresponding to the previous video frame, and the model parameters include parameters of an update gate in the convolutional gated loop unit;

[0145] The parameters of the update gate are modified according to the second loss function value.

[0146] The video segmentation model training device provided in the embodiments of the present disclosure can execute the video segmentation model training method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0147] Figure 7 This is a structural diagram of a video segmentation device provided in an embodiment of the present disclosure. The device can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server.

[0148] like Figure 7 As shown, the device includes: a second video acquisition module 710 , a second mask determination module 720 and an object segmentation module 730 .

[0149] A second video acquisition module 710 is used to acquire a video to be segmented, wherein the video frame of the video to be segmented includes a bust of the target object;

[0150] A second mask determination module 720 is used to input the video to be segmented into a video segmentation model, and output an object mask corresponding to the target object in the video to be segmented through the video segmentation model, wherein the video segmentation model is trained using the training method of the video segmentation model described in any embodiment of the present disclosure;

[0151] The object segmentation module 730 is used to segment the target object in the video to be segmented according to the object mask corresponding to the target object in the video to be segmented.

[0152] The video segmentation device provided in the embodiments of the present disclosure can execute the video segmentation method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0153] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0154] Figure 8 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 8 , which shows an electronic device (eg, Figure 8The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0155] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An edit / output (I / O) interface 805 is also connected to the bus 804.

[0156] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0157] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0158] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0159] The electronic device provided by the embodiment of the present disclosure and the method provided by the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0160] The embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the method provided by the above embodiments is implemented.

[0161] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0162] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0163] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0164] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains sample pictures and sample videos, wherein the sample pictures and sample videos include a bust of a target object;

[0165] Adjusting the pixel position corresponding to the target object in the sample image to obtain a simulated video, and determining a training sample set according to the sample video and the simulated video;

[0166] Inputting the video frames in the training sample set into a video segmentation model to be trained, and determining a predicted mask of a target object in the video frame by using the video segmentation model;

[0167] A loss function value is determined according to a predicted mask and a target mask of the video frame, and a model parameter of the video segmentation model is corrected according to the loss function value, wherein the target mask represents the target object in the video frame.

[0168] Alternatively, the computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device: obtains a video to be segmented, wherein the video frame of the video to be segmented includes a bust of the target object;

[0169] Inputting the video to be segmented into a video segmentation model, and outputting an object mask corresponding to the target object in the video to be segmented through the video segmentation model, wherein the video segmentation model is trained using the training method of the video segmentation model described in any embodiment of the present disclosure;

[0170] The target object in the video to be segmented is segmented according to the object mask corresponding to the target object in the video to be segmented.

[0171] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0172] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0173] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0174] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0175] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0176] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0177] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0178] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A video segmentation model training method, characterized in that: include: Acquire sample pictures and sample videos, wherein the sample pictures and sample videos include a bust of the target object; Adjusting the pixel position corresponding to the target object in the sample image to obtain a simulated video, and determining a training sample set according to the sample video and the simulated video; Inputting the video frames in the training sample set into a video segmentation model to be trained, and determining a predicted mask of a target object in the video frame by using the video segmentation model; A loss function value is determined according to a predicted mask and a target mask of the video frame, and a model parameter of the video segmentation model is corrected according to the loss function value, wherein the target mask represents the target object in the video frame.

2. The method according to claim 1, characterized in that The step of adjusting the pixel position corresponding to the target object in the sample image to obtain a simulated video includes: Performing N displacement operations on pixels corresponding to the target object in the sample image along the target direction to obtain N target images; The sample image and N target images are combined according to the shift operation sequence to obtain a simulated video.

3. The method according to claim 1, characterized in that The step of determining a training sample set according to the sample video and the simulated video includes: For a video frame in the sample video or the simulated video, determining timing information of the video frame and a target mask of a target object; Adding the timing information to a feature tensor corresponding to the video frame to obtain a target feature tensor corresponding to the video frame, wherein the target feature tensor is used to characterize the video frame; A training sample set is determined according to the target feature tensor and the target mask.

4. The method according to claim 3, characterized in that The step of determining the timing information of the video frame and the target mask of the target object for the video frame in the sample video or the simulated video includes: Segmenting the sample video and the simulation video to obtain video clips; For a video frame in the video segment, determining timing information of the video frame according to a position of the video frame in the video segment; Performing target object recognition on the video frame using at least two pre-trained recognition models to obtain at least two candidate masks of the target object in the video frame; The overlap degree is determined according to the at least two candidate masks, and the target mask of the target object in the video frame is determined according to the overlap degree and the at least two candidate masks.

5. The method according to claim 4, characterized in that The step of determining the overlap degree according to the at least two candidate masks, and determining the target mask of the target object in the video frame according to the overlap degree and the at least two candidate masks, comprises: Determining a degree of overlap based on the intersection and union of the at least two candidate masks; If the overlap degree is greater than a set overlap threshold, a target mask of the target object in the video frame is determined according to any candidate mask of the at least two candidate masks.

6. The method according to claim 1, characterized in that The step of inputting the video frames in the training sample set into the video segmentation model to be trained, and determining the predicted mask of the target object in the video frame by using the video segmentation model, comprises: Inputting video frames of the sample video or the simulated video into the video segmentation model to be trained according to a target sampling ratio, wherein the target sampling ratio represents a ratio of the number of times the sample video is used to train the video segmentation model to the number of times the simulated video is used to train the video segmentation model during at least two model iterations; Obtain a predicted mask of a target object in the video frame output by the video segmentation model.

7. The method according to claim 1, characterized in that The step of determining a loss function value according to the predicted mask and the target mask of the video frame, and correcting a model parameter of the video segmentation model according to the loss function value, comprises: A loss function value is determined according to the predicted mask, the target mask of the video frame and the target mask of the previous video frame, and a model parameter of the video segmentation model is corrected according to the loss function value.

8. The method according to claim 7, characterized in that The step of determining a loss function value according to the predicted mask of the video frame, the target mask, and the target mask of the previous video frame, and correcting the model parameters of the video segmentation model according to the loss function value includes: Determine a prediction deviation according to a prediction mask and a target mask of the video frame, and determine a first loss function value according to the prediction deviation; Determine a target deviation according to a target mask of the video frame and a target mask of a previous video frame, and determine a second loss function value according to the target deviation; Correcting model parameters of the video segmentation model according to the first loss function value, wherein the decoder of the video segmentation model includes at least one convolutional gated loop unit, the convolutional gated loop unit is used to determine the hidden layer features of the video frame according to the hidden layer features corresponding to the previous video frame, and the model parameters include parameters of an update gate in the convolutional gated loop unit; The parameters of the update gate are modified according to the second loss function value.

9. A video segmentation method, characterized in that: include: Acquire a video to be segmented, wherein a video frame of the video to be segmented includes a bust of a target object; Inputting the video to be segmented into a video segmentation model, and outputting an object mask corresponding to the target object in the video to be segmented through the video segmentation model, wherein the video segmentation model is trained using the method according to any one of claims 1 to 8; The target object in the video to be segmented is segmented according to the object mask corresponding to the target object in the video to be segmented.

10. A video segmentation model training device, characterized in that: include: A first video acquisition module, used to acquire sample pictures and sample videos, wherein the sample pictures and sample videos include a bust of a target object; A sample determination module, used to adjust the pixel position corresponding to the target object in the sample image to obtain a simulated video, and determine a training sample set according to the sample video and the simulated video; A first mask determination module, configured to input the video frame in the training sample set into a video segmentation model to be trained, and determine a predicted mask of a target object in the video frame by using the video segmentation model; A parameter correction module is used to determine a loss function value according to a predicted mask and a target mask of the video frame, and to correct model parameters of the video segmentation model according to the loss function value, wherein the target mask represents the target object in the video frame.

11. A video segmentation device, characterized in that: include: A second video acquisition module is used to acquire a video to be segmented, wherein the video frame of the video to be segmented includes a bust of the target object; A second mask determination module, configured to input the video to be segmented into a video segmentation model, and output an object mask corresponding to the target object in the video to be segmented through the video segmentation model, wherein the video segmentation model is trained using the method according to any one of claims 1 to 8; The object segmentation module is used to segment the target object in the video to be segmented according to the object mask corresponding to the target object in the video to be segmented.

12. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

13. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to perform the method according to any one of claims 1 to 9 when executed by a computer processor.

Images