An image processing method, apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to an image processing method, apparatus, electronic device, and storage medium. The method includes: extracting features from video frame images to obtain initial segmentation region features corresponding to each video frame image. In a segmentation feature processing network, performing feature processing on the initial segmentation region features based on a first preset order and performing feature processing on the initial segmentation region features based on a second preset order to obtain target segmentation region features. The first preset order is the forward chronological order corresponding to a preset video, and the second preset order is the reverse chronological order corresponding to the preset video. Performing image segmentation processing through the target segmentation region features to obtain an image segmentation result corresponding to a target object. This method can refer to the region classification features corresponding to the video frame images before and after each video frame image during the region segmentation process, thereby improving the stability and accuracy of the image segmentation result.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to an image processing method, apparatus, electronic device, and storage medium. Background Art

[0002] Image segmentation technology can perform binary classification on each pixel point in the image to be segmented, and assign each pixel point to the target segmentation area and the background area, so that the target object can be obtained from the image to be segmented. Image segmentation technology is widely used in image processing, video processing, etc. In the related art, disassembling the video frame images in a preset video and inputting each video frame image into a model for feature extraction and image segmentation will result in unstable image segmentation results and prone to flashing problems. Summary of the Invention

[0003] The present disclosure provides an image processing method, apparatus, electronic device, and storage medium to at least solve the problem of unstable image segmentation results and prone to flashing in the related art. The technical solution of the present disclosure is as follows:

[0004] According to the first aspect of the embodiments of the present disclosure, an image processing method is provided, and the method includes:

[0005] Obtain a preset number of video frame images corresponding to the target object;

[0006] Extract features from the video frame images to obtain initial segmentation region features corresponding to each video frame image;

[0007] Based on the first preset order and the second preset order, input the initial segmentation region features into a segmentation feature processing network for feature processing to obtain target segmentation region features; the first preset order is the forward chronological order corresponding to the preset video, and the second preset order is the reverse chronological order corresponding to the preset video;

[0008] Based on the target segmentation region features, perform image segmentation processing on each video frame image to obtain an image segmentation result corresponding to the target object.

[0009] As an optional embodiment, the segmentation feature processing network includes an associated feature determination layer and a feature splicing layer, and the inputting the initial segmentation region features into the segmentation feature processing network for feature processing based on the first preset order and the second preset order to obtain the target segmentation region features includes:

[0010] Based on the first preset order, use the initial segmentation region features corresponding to the video frame images before each video frame image as the first associated segmentation region features;

[0011] Input the initial segmentation region features corresponding to each of the video frame images and the first associated segmentation region features into the associated feature determination layer to determine associated features, obtaining the first associated features corresponding to each of the video frame images;

[0012] Based on the second preset order, use the initial segmentation region features corresponding to the video frame images after each of the video frame images as the second associated segmentation region features;

[0013] Input the initial segmentation region features corresponding to each of the video frame images and the second associated segmentation region features into the associated feature determination layer to determine associated features, obtaining the second associated features corresponding to each of the video frame images;

[0014] Input the first associated features and the second associated features into the feature splicing layer to perform feature splicing, obtaining the target segmentation region features.

[0015] As an optional embodiment, the step of inputting the initial segmentation region features corresponding to each of the video frame images and the first associated segmentation region features into the associated feature determination layer to determine associated features, obtaining the first associated features corresponding to each of the video frame images includes:

[0016] Obtain the first region classification feature corresponding to the current video frame image; the first region classification feature is obtained by inputting the first associated segmentation region feature corresponding to the current video frame image into the associated feature determination layer;

[0017] Input the initial segmentation region feature corresponding to the current video frame image and the first region classification feature corresponding to the current video frame image into the associated feature determination layer to perform feature processing, obtaining the first associated feature corresponding to the current video frame image;

[0018] Use the first associated feature as the first region classification feature of the next video frame image of the current video frame image, and use the next video frame image as the new current video frame image, and return to execute the step of obtaining the first region classification feature corresponding to the current video frame image; the next video frame image is the next video frame image of the current video frame image in the first preset order.

[0019] As an optional embodiment, the step of inputting the initial segmentation region features corresponding to each of the video frame images and the second associated segmentation region features into the associated feature determination layer to determine associated features, obtaining the second associated features corresponding to each of the video frame images includes:

[0020] Obtain the second region classification feature corresponding to the current video frame image; the second region classification feature is obtained by inputting the second associated segmentation region feature corresponding to the current video frame image into the associated feature determination layer;

[0021] Input the initial segmentation region feature corresponding to the current video frame image and the second region classification feature corresponding to the current video frame image into the associated feature determination layer for feature processing to obtain the second associated feature corresponding to the current video frame image;

[0022] Use the second associated feature as the second region classification feature of the next video frame image of the current video frame image, and use the next video frame image as the new current video frame image, and return to execute the step of obtaining the second region classification feature corresponding to the current video frame image; the next video frame image is the next video frame image of the current video frame image in the second preset order.

[0023] As an optional embodiment, the obtaining of the preset number of video frame images corresponding to the target object includes:

[0024] Identify video clip segments from a preset video, and each video clip segment is a video segment between two adjacent clip points in the preset video;

[0025] Use the video frame images in each video clip segment as the preset number of video frame images.

[0026] As an optional embodiment, the method further includes:

[0027] Obtain a preset number of training video frame images corresponding to the training object and the training object annotation information for each training video frame image;

[0028] Extract features from the training video frame images to obtain the initial segmentation region training features corresponding to each training video frame image;

[0029] Based on the first training order and the second training order, input the initial segmentation region training features into the network to be trained for feature processing to obtain the target segmentation region training features; the first training order is the forward chronological order corresponding to the training video, and the second training order is the reverse chronological order corresponding to the training video;

[0030] Train the network to be trained based on the target segmentation region training features and the training object annotation information to obtain the segmentation feature processing network.

[0031] As an optional embodiment, training the network to be trained based on the training features of the target segmentation region and the labeled information of the training object to obtain the segmentation feature processing network includes:

[0032] Obtaining segmentation loss information based on the training features of the target segmentation region and the labeled information of the training object;

[0033] Training the network to be trained based on the segmentation loss information to obtain the segmentation feature processing network.

[0034] According to the second aspect of the embodiments of the present disclosure, there is provided an image processing apparatus, the apparatus includes;

[0035] A video frame image acquisition module, configured to acquire a preset number of video frame images corresponding to a target object;

[0036] A feature extraction module, configured to perform feature extraction on the video frame images to obtain initial segmentation region features corresponding to each video frame image;

[0037] A feature processing module, configured to perform feature processing on the initial segmentation region features by inputting them into a segmentation feature processing network based on a first preset order and a second preset order to obtain target segmentation region features; the first preset order is the forward time sequence order corresponding to the preset video, and the second preset order is the reverse time sequence order corresponding to the preset video;

[0038] An image segmentation module, configured to perform image segmentation processing on each video frame image based on the target segmentation region features to obtain an image segmentation result corresponding to the target object.

[0039] As an optional embodiment, the segmentation feature processing network includes an associated feature determination layer and a feature splicing layer, and the feature processing module includes:

[0040] A first associated segmentation region feature determination unit, configured to perform taking the initial segmentation region features corresponding to the video frame images before each video frame image as first associated segmentation region features based on the first preset order;

[0041] A first associated feature determination unit, configured to perform inputting the initial segmentation region features corresponding to each video frame image and the first associated segmentation region features into the associated feature determination layer to perform associated feature determination to obtain a first associated feature corresponding to each video frame image;

[0042] The second associated segmentation region feature determination unit is configured to execute, based on the second preset order, using the initial segmentation region feature corresponding to the video frame image after each of the video frame images as the second associated segmentation region feature;

[0043] The second associated feature determination unit is configured to execute inputting the initial segmentation region feature corresponding to each of the video frame images and the second associated segmentation region feature into the associated feature determination layer to determine the associated feature, and obtain the second associated feature corresponding to each of the video frame images;

[0044] The feature splicing unit is configured to execute inputting the first associated feature and the second associated feature into the feature splicing layer to perform feature splicing, and obtain the target segmentation region feature.

[0045] As an optional embodiment, the first associated feature determination unit includes:

[0046] The first region classification feature acquisition unit is configured to execute acquiring the first region classification feature corresponding to the current video frame image; the first region classification feature is obtained by inputting the first associated segmentation region feature corresponding to the current video frame image into the associated feature determination layer;

[0047] The first associated feature acquisition unit is configured to execute inputting the initial segmentation region feature corresponding to the current video frame image and the first region classification feature corresponding to the current video frame image into the associated feature determination layer to perform feature processing, and obtain the first associated feature corresponding to the current video frame image;

[0048] The first video frame image update unit is configured to execute using the first associated feature as the first region classification feature of the next video frame image of the current video frame image, and using the next video frame image as the new current video frame image, and returning to execute the step of acquiring the first region classification feature corresponding to the current video frame image; the next video frame image is the next video frame image of the current video frame image in the first preset order.

[0049] As an optional embodiment, the second associated feature determination unit includes:

[0050] The second region classification feature acquisition unit is configured to execute acquiring the second region classification feature corresponding to the current video frame image; the second region classification feature is obtained by inputting the second associated segmentation region feature corresponding to the current video frame image into the associated feature determination layer;

[0051] A second associated feature obtaining unit, configured to perform inputting the initial segmentation region feature corresponding to the current video frame image and the second region classification feature corresponding to the current video frame image into the associated feature determination layer for feature processing, to obtain the second associated feature corresponding to the current video frame image;

[0052] A second video frame image updating unit, configured to perform using the second associated feature as the second region classification feature of the next video frame image of the current video frame image, and using the next video frame image as the new current video frame image, and returning to execute the step of obtaining the second region classification feature corresponding to the current video frame image; the next video frame image is the next video frame image of the current video frame image in the second preset order.

[0053] As an optional embodiment, the video frame image obtaining module includes:

[0054] A clip segment recognition unit, configured to perform clip segment recognition on a preset video to obtain a plurality of video clip segments, and each video clip segment is a video segment between two adjacent clip points in the preset video;

[0055] A video frame image determination unit, configured to perform using the video frame images in each video clip segment as the preset number of video frame images.

[0056] As an optional embodiment, the device further includes:

[0057] A training data obtaining module, configured to perform obtaining a preset number of training video frame images corresponding to a training object and training object annotation information for each training video frame image;

[0058] A training feature extraction module, configured to perform feature extraction on the training video frame images to obtain initial segmentation region training features corresponding to each training video frame image;

[0059] A training feature processing module, configured to perform based on a first training order and a second training order, inputting the initial segmentation region training features into a network to be trained for feature processing, to obtain target segmentation region training features; the first training order is the forward chronological order corresponding to the training video, and the second training order is the reverse chronological order corresponding to the training video;

[0060] A network training module, configured to perform training the network to be trained based on the target segmentation region training features and the training object annotation information, to obtain the segmentation feature processing network.

[0061] As an optional embodiment, the network training module includes:

[0062] A segmentation loss determination unit configured to execute obtaining segmentation loss information based on the training features of the target segmentation region and the annotation information of the training object;

[0063] A network training unit configured to execute training the network to be trained based on the segmentation loss information to obtain the segmentation feature processing network.

[0064] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0065] A processor;

[0066] A memory for storing executable instructions of the processor;

[0067] Wherein, the processor is configured to execute the instructions to implement the image processing method as described above.

[0068] According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enabling the electronic device to execute the image processing method as described above.

[0069] According to a fifth aspect of an embodiment of the present disclosure, there is provided a computer program product, including a computer program, where the computer program implements the image processing method as described above when executed by a processor.

[0070] The technical solution provided by the embodiment of the present disclosure at least brings the following beneficial effects:

[0071] Feature extraction is performed on video frame images to obtain initial segmentation region features corresponding to each video frame image. In the segmentation feature processing network, feature processing is performed on the initial segmentation region features based on a first preset order and feature processing is performed on the initial segmentation region features based on a second preset order to obtain target segmentation region features. The first preset order is the forward time sequence order corresponding to a preset video, and the second preset order is the reverse time sequence order corresponding to the preset video. Image segmentation processing is performed through the target segmentation region features to obtain an image segmentation result corresponding to the target object. This method can refer to the feature recognition information corresponding to the video frame images before and after each video frame image during the region segmentation process, thereby improving the stability and accuracy of the image segmentation result.

[0072] It should be understood that the above general description and subsequent detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0074] Figure 1 It is a schematic diagram of an application scenario of an image processing method shown according to an exemplary embodiment.

[0075] Figure 2 It is a flowchart of an image processing method shown according to an exemplary embodiment.

[0076] Figure 3 It is a flowchart of obtaining a preset number of video frame images in an image processing method shown according to an exemplary embodiment.

[0077] Figure 4 It is a schematic structural diagram of a segmentation feature encoding network and a segmentation feature decoding network in an image processing method shown according to an exemplary embodiment.

[0078] Figure 5 It is a flowchart of obtaining target segmentation region features in an image processing method shown according to an exemplary embodiment.

[0079] Figure 6 It is a flowchart of obtaining a first associated feature corresponding to each video frame image in an image processing method shown according to an exemplary embodiment.

[0080] Figure 7 It is a schematic diagram of forward processing and reverse processing of video frame images in an image processing method shown according to an exemplary embodiment.

[0081] Figure 8 It is a flowchart of obtaining a second associated feature corresponding to each video frame image in an image processing method shown according to an exemplary embodiment.

[0082] Figure 9 It is a flowchart of model training in an image processing method shown according to an exemplary embodiment.

[0083] Figure 10 It is a flowchart of model training based on segmentation loss information in an image processing method shown according to an exemplary embodiment.

[0084] Figure 11 It is a block diagram of an image processing apparatus shown according to an exemplary embodiment.

[0085] Figure 12 It is a block diagram of an electronic device on the server side shown according to an exemplary embodiment. Detailed implementation manners

[0086] In order to enable those of ordinary skill in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0087] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described here can be implemented in an order other than those illustrated or described here. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0088] Figure 1 is a schematic diagram of an application scenario of an image processing method shown according to an exemplary embodiment. The application scenario includes a client 110 and a server 120. The server 120 can obtain a preset video from the client 110. The server 120 obtains the initial segmentation region features of each video frame image in the preset video through a feature decoding network. The server 120 processes the initial segmentation region features based on a first preset order and processes the initial segmentation region features based on a second preset order to obtain target segmentation region features. The server 120 performs image segmentation based on the target segmentation region features and sends the image segmentation result to the client 110.

[0089] In the embodiments of the present disclosure, the client 110 includes entity devices of types such as smart phones, desktop computers, tablet computers, laptop computers, digital assistants, smart wearable devices, etc., and may also include software running on the entity devices, such as application programs, etc. The operating systems running on the entity devices in the embodiments of the present application may include, but are not limited to, Android systems, IOS systems, linux, Unix, windows, etc.

[0090] In the embodiments of the present disclosure, the server 120 may include an independently operating server, or a distributed server, or a server cluster composed of multiple servers. The server 120 may include a network communication unit, a processor, a memory, and so on.

[0091] Figure 2 is a flowchart of an image processing method shown according to an exemplary embodiment, as Figure 2 shown, this method is used in a server and includes the following steps.

[0092] S210. Obtain a preset number of video frame images corresponding to a target object;

[0093] As an optional embodiment, the preset number of video frame images can be the video frame images in a preset video, or can be the video frame images in a video clip segment after the preset video is divided into multiple video clip segments.

[0094] As an optional embodiment, please refer to Figure 3 , obtaining the preset number of video frame images corresponding to the target object includes:

[0095] [[ID=IO]]S310. Perform clip segment recognition on the preset video to obtain multiple video clip segments, and each video clip segment is a video segment between two adjacent clip points in the preset video;

[0096] S320. Use the video frame images in each video clip segment as the preset number of video frame images.

[0097] As an optional embodiment, clip segment recognition can be performed on the preset video based on a preset clip segment recognition network. Input the preset video into the clip segment recognition network for clip segment recognition to obtain multiple video clip segments. The clip point is used to represent the boundary information of the video clip segment. In the clip segment recognition network, clip mutation detection can be performed on the preset video to obtain the first clip point in the preset video, and clip gradual change detection can be performed on the preset video to obtain the second clip point in the preset video, so as to use the video segment between two adjacent clip points as the video clip segment.

[0098] Clip mutation detection can determine the difference information between two video frame images, compare the difference information between two video frame images with a preset difference threshold. When the difference information is greater than the preset difference threshold, it is determined that there is a first clip point between the two video frame images. When the difference information is less than the preset difference threshold, it is determined that there is no first clip point between the two video frame images. The preset difference threshold can be an adaptive difference threshold, and the adaptive difference threshold can adjust the threshold according to different types of preset videos. The methods of clip mutation detection can include frame difference detection, histogram segmentation, block-based shot segmentation, motion vectors, and other methods. When performing clip mutation detection, flash detection can be performed on the preset video to determine the non-clip mutation points in the preset video. The non-clip mutation points can be the image mutation positions corresponding to the light changes in the preset video. Remove the non-clip mutation points from the detected first clip points to obtain the third clip points. Determine the video clip segments through the third clip points.

[0099] It should be noted that in the above translation, "IO" in line should be "S310" according to the original text content, which may be a small error in the original text. The translation is adjusted accordingly.Clip fade detection can determine the continuous change process between two video clip segments, such as fade-in, fade-out, dissolve, etc. When performing clip fade detection, image grayscale detection can be performed on a preset video to determine the non-clip fade points in the preset video. The non-clip fade points can be the image fade positions corresponding to the object dynamics in the preset video. Remove the non-clip fade points from the detected second clip points to obtain the fourth clip points. Determine the video clip segments through the fourth clip points. The methods of clip fade detection can include various methods such as motion sequence models, edge shot detection, B-spline detection, etc.

[0100] By identifying the video clip segments in the preset video, it is possible to avoid the problems of incorrect video frame images in terms of timing before and after clip and easy flickering of the image segmentation results due to clip, thereby improving the stability and accuracy of the image segmentation results at the clip points.

[0101] S220. Extract features from the video frame images to obtain the initial segmentation region features corresponding to each video frame image;

[0102] As an optional embodiment, input the video frame images into an image segmentation network for feature extraction, and the initial segmentation region features corresponding to each video frame image can be obtained. In the encoder-decoder structure, the image segmentation network can be a segmentation feature encoding network, and the segmentation feature processing network can be a segmentation feature decoding network. Please refer to Figure 4 as Figure 4 shown in the schematic structural diagrams of the segmentation feature encoding network and the segmentation feature decoding network. Input the video frame images into the segmentation feature encoding network to perform feature encoding on the information in the video frame images, that is, identify the pixel points corresponding to the target object regions and the pixels corresponding to the image background regions in the video frame images, so as to segment the target object regions and the image background regions, and obtain the initial segmentation region features corresponding to each video frame image.

[0103] S230. Based on the first preset order and the second preset order, input the initial segmentation region features into the segmentation feature processing network for feature processing to obtain the target segmentation region features; the first preset order is the forward time sequence order corresponding to the preset video, and the second preset order is the reverse time sequence order corresponding to the preset video;

[0104] As an optional embodiment, in the encoder-decoder structure, the image segmentation network can be a segmentation feature encoding network, and the segmentation feature processing network can be a segmentation feature decoding network. Please refer to Figure 4 as Figure 4The figure shows a schematic structural diagram of a segmentation feature encoding network and a segmentation feature decoding network. In the segmentation feature processing network, an associated feature determination layer is provided. The initial segmentation region features are input into the associated feature determination layer in the first preset order for feature processing, and the first associated feature can be obtained. The initial segmentation region features are input into the associated feature determination layer in the second preset order for feature processing, and the second associated feature can be obtained. In the segmentation feature processing network, the first associated feature and the second associated feature are spliced to obtain the target segmentation region features.

[0105] As an optional embodiment, please refer to Figure 5 , the segmentation feature processing network includes an associated feature determination layer and a feature splicing layer. Based on the first preset order and the second preset order, the initial segmentation region features are input into the segmentation feature processing network for feature processing, and obtaining the target segmentation region features includes:

[0106] S510. Based on the first preset order, the initial segmentation region features corresponding to the video frame images before each video frame image are used as the first associated segmentation region features;

[0107] S520. The initial segmentation region features corresponding to each video frame image and the first associated segmentation region features are input into the associated feature determination layer for associated feature determination to obtain the first associated features corresponding to each video frame image;

[0108] S530. Based on the second preset order, the initial segmentation region features corresponding to the video frame images after each video frame image are used as the second associated segmentation region features;

[0109] S540. The initial segmentation region features corresponding to each video frame image and the second associated segmentation region features are input into the associated feature determination layer for associated feature determination to obtain the second associated features corresponding to each video frame image;

[0110] S550. The first associated feature and the second associated feature are input into the feature splicing layer for feature splicing to obtain the target segmentation region features.

[0111] As an optional embodiment, the previous video frame image of each video frame image determined by the first preset order is used as the first preset video frame image. Based on the first preset order, the initial segmentation region features corresponding to the first video frame image to the first preset video frame image are used as the first associated segmentation region features.

[0112] Input the initial segmentation region features and the first associated segmentation region features corresponding to each video frame image into the associated feature determination layer for associated feature determination. In the case where the input is the first video frame image, only the initial segmentation region features corresponding to the first video frame image can be input into the associated feature determination layer for associated feature determination.

[0113] Through the associated feature determination layer, when inputting in the forward order to the first preset video frame image, the implicit state association information corresponding to the first associated segmentation region features can be recognized. By performing feature processing on the initial segmentation region features corresponding to each video frame image and the implicit state association information corresponding to the first associated segmentation region features, the first associated features corresponding to each video frame image can be obtained.

[0114] Take the previous video frame image of each video frame image determined by the second preset order as the second preset video frame image. Based on the second preset order, take the initial segmentation region features corresponding to each video frame image from the last video frame image to the second preset video frame image as the second associated segmentation region features.

[0115] Input the initial segmentation region features and the second associated segmentation region features corresponding to each video frame image into the associated feature determination layer for associated feature determination. In the case where the input is the last video frame image, only the initial segmentation region features corresponding to the last video frame image can be input into the associated feature determination layer for associated feature determination.

[0116] Through the associated feature determination layer, when inputting in the reverse order to the second preset video frame image, the implicit state association information corresponding to the second associated segmentation region features can be recognized. By performing feature processing on the initial segmentation region features corresponding to each video frame image and the implicit state association information corresponding to the second associated segmentation region features, the second associated features corresponding to each video frame image can be obtained.

[0117] Among them, the implicit state association information can represent the feature recognition information for segmenting the target object region and the image background region, that is, the feature recognition information for determining whether the pixel points in the video frame image are pixel points corresponding to the target object region or pixel points corresponding to the image background region.

[0118] Input the first associated features and the second associated features into the feature splicing layer, and perform splicing processing on the first associated features and the second associated features to obtain the target segmentation region features. The target segmentation region features include the implicit state association information determined based on the forward time sequence order and also include the implicit state association information determined based on the reverse time sequence order.

[0119] Determining the first associated feature based on the first preset order and determining the second associated feature based on the second preset order can increase the feature recognition information for reference during image segmentation, thereby improving the accuracy of the image segmentation result.

[0120] As an optional embodiment, please refer to Figure 6 , input the initial segmentation region feature and the first associated segmentation region feature corresponding to each video frame image into the associated feature determination layer to determine the associated feature, and the first associated feature corresponding to each video frame image includes:

[0121] S610. Obtain the first region classification feature corresponding to the current video frame image; the first region classification feature is obtained by inputting the first associated segmentation region feature corresponding to the current video frame image into the associated feature determination layer;

[0122] S620. Input the initial segmentation region feature corresponding to the current video frame image and the first region classification feature corresponding to the current video frame image into the associated feature determination layer for feature processing to obtain the first associated feature corresponding to the current video frame image;

[0123] S630. Use the first associated feature as the first region classification feature of the next video frame image of the current video frame image, and use the next video frame image as the new current video frame image, and return to execute the step of obtaining the first region classification feature corresponding to the current video frame image; the next video frame image is the next video frame image of the current video frame image in the first preset order.

[0124] As an optional embodiment, the hidden state association information may include the first region classification feature. In the region classification feature determination layer, feature processing is performed on the initial segmentation region feature corresponding to the first video frame image to obtain the first associated feature corresponding to the first video frame image. Use the initial segmentation region feature corresponding to the first video frame image as the first associated segmentation region feature corresponding to the second video frame image, then the first associated feature corresponding to the first video frame image is the first region classification feature corresponding to the second video frame image. Feature processing is performed on the initial segmentation region feature corresponding to the second video frame image and the first region classification feature corresponding to the second video frame image to obtain the first associated feature corresponding to the second video frame image. Use the initial segmentation region features corresponding to the first video frame image and the second video frame image respectively as the first associated segmentation region feature corresponding to the third video frame image, then the first associated feature corresponding to the second video frame image is the first region classification feature corresponding to the third video frame image. Feature processing is performed on the initial segmentation region feature corresponding to the third video frame image and the first region classification feature corresponding to the third video frame image to obtain the first associated feature corresponding to the third video frame image.

[0125] By analogy, taking the initial segmentation region features corresponding to the video frame images before each video frame image as the first associated segmentation region features corresponding to each video frame image, the first associated features corresponding to the previous video frame image of each video frame image can be used as the first region classification features corresponding to each video frame image. Then, the initial segmentation region features corresponding to each video frame image and the first region classification features corresponding to each video frame image are processed to obtain the first associated features corresponding to each video frame image.

[0126] Please refer to Figure 7 , such as Figure 7 shown in the schematic diagram of the forward processing and reverse processing of video frame images. A preset number of video frame images are numbered as video frame image 1 to video frame image T in the forward time sequence order. Based on the first preset order, the initial segmentation region feature X1 corresponding to video frame image 1 is processed to obtain the first associated feature H1. The first associated feature H1 is used as the first region classification feature corresponding to video frame image 2. The initial segmentation region feature X2 corresponding to video frame image 2 and the first region classification feature corresponding to video frame image 2 are processed to obtain the first associated feature H2 corresponding to video frame image 2. The first associated feature H2 is used as the first region classification feature corresponding to video frame image 3. The initial segmentation region feature X3 corresponding to video frame image 3 and the first region classification feature corresponding to video frame image 3 are processed to obtain the first associated feature H3 corresponding to video frame image 3. By analogy, the first associated feature H T-1 is used as the first region classification feature corresponding to video frame image T. The initial segmentation region feature X T corresponding to video frame image T and the first region classification feature corresponding to video frame image T are processed to obtain the first associated feature H T .

[0127] Based on the first preset order to determine the associated features of the initial segmentation region features, the feature recognition information in the video frame images before the video frame image can be obtained, so that the image segmentation can be performed by referring to the feature recognition information, improving the accuracy of the image segmentation result.

[0128] As an optional embodiment, please refer to Figure 8 , inputting the initial segmentation region features corresponding to each video frame image and the second associated segmentation region features into the associated feature determination layer for associated feature determination, and obtaining the second associated features corresponding to each video frame image includes:

[0129] S810. Obtain the second region classification feature corresponding to the current video frame image; the second region classification feature is obtained by inputting the second associated segmentation region feature corresponding to the current video frame image into the associated feature determination layer;

[0130] S830. Input the initial segmentation region feature corresponding to the current video frame image and the second region classification feature corresponding to the current video frame image into the associated feature determination layer for feature processing to obtain the second associated feature corresponding to the current video frame image;

[0131] S830. Use the second associated feature as the second region classification feature of the next video frame image of the current video frame image, and use the next video frame image as the new current video frame image, and return to execute the step of obtaining the second region classification feature corresponding to the current video frame image; the next video frame image is the next video frame image of the current video frame image in the second preset order.

[0132] As an optional embodiment, the hidden state association information may include the second region classification feature. In the region classification feature determination layer, feature processing is performed on the initial segmentation region feature corresponding to the last video frame image to obtain the second associated feature corresponding to the last video frame image. Use the initial segmentation region feature corresponding to the last video frame image as the second associated segmentation region feature corresponding to the penultimate video frame image, then the second associated feature corresponding to the last video frame image is the second region classification feature corresponding to the penultimate video frame image. Perform feature processing on the initial segmentation region feature corresponding to the penultimate video frame image and the second region classification feature corresponding to the penultimate video frame image to obtain the second associated feature corresponding to the penultimate video frame image. Use the initial segmentation region features corresponding to the last video frame image and the penultimate video frame image respectively as the second associated segmentation region feature corresponding to the third-to-last video frame image, then the second associated feature corresponding to the penultimate video frame image is the second region classification feature corresponding to the third-to-last video frame image. Perform feature processing on the initial segmentation region feature corresponding to the third-to-last video frame image and the second region classification feature corresponding to the third-to-last video frame image to obtain the second associated feature corresponding to the third-to-last video frame image. [[ID=X]] [[ID=Y]]

[0133] By analogy, use the initial segmentation region feature corresponding to the video frame image after each video frame image as the second associated segmentation region feature corresponding to each video frame image, then the second associated feature corresponding to the next video frame image of each video frame image can be used as the second region classification feature corresponding to each video frame image, and then perform feature processing on the initial segmentation region feature corresponding to each video frame image and the second region classification feature corresponding to each video frame image to obtain the second associated feature corresponding to each video frame image.

[0134] Please refer to Figure 7 , as Figure 7 shown in the schematic diagram of the forward processing and reverse processing of video frame images. A preset number of video frame images are numbered as video frame image 1 to video frame image T in the forward time sequence order. Based on the second preset order, the initial segmentation region feature X T corresponding to video frame image T is subjected to feature processing to obtain the second associated feature H T '. The second associated feature H T ' is used as the first region classification feature corresponding to video frame image T - 1. The initial segmentation region feature X T-1 corresponding to video frame image T - 1 and the first region classification feature corresponding to video frame image T - 1 are subjected to feature processing to obtain the second associated feature H T-1 ' corresponding to video frame image T. The second associated feature H T-1 ' is used as the first region classification feature corresponding to video frame image T - 2. The initial segmentation region feature X T-2 corresponding to video frame image T - 2 and the first region classification feature corresponding to video frame image T - 2 are subjected to feature processing to obtain the second associated feature H T-2 ' corresponding to video frame image T - 2. And so on, the second associated feature H2' is used as the first region classification feature corresponding to video frame image 1. The initial segmentation region feature X1 corresponding to video frame image 1 and the first region classification feature corresponding to video frame image 1 are subjected to feature processing to obtain the second associated feature H1' corresponding to video frame image 1.

[0135] Based on the second preset order to determine the associated feature of the initial segmentation region feature, the feature recognition information in the video frame image after the video frame image can be obtained, so that the image segmentation can be referred to this feature recognition information, improving the accuracy of the image segmentation result.

[0136] S240. Based on the target segmentation region feature, perform image segmentation processing on each video frame image to obtain the image segmentation result corresponding to the target object.

[0137] [[ID=~30]]As an optional embodiment, the target segmentation region feature may be a mask of the target object region. Based on the target segmentation region feature, image segmentation processing can be performed on each video frame image to obtain the image segmentation result corresponding to the target object. Based on this image segmentation result, video background replacement, video special effect processing, virtual background generation, etc. can be performed on the preset video.

[0138] As an optional embodiment, please refer to Figure 9 , the method further includes:

[0139] S910. Obtaining a preset number of training video frame images corresponding to the training object and training object annotation information for each training video frame image;

[0140] S920. Perform feature extraction on the training video frame image to obtain the initial segmentation region training features corresponding to each training video frame image;

[0141] S930. Based on the first training sequence and the second training sequence, the initial segmentation region training features are input into the to-be-trained network for feature processing to obtain the target segmentation region training features; the first training sequence is the forward time sequence corresponding to the training video, and the second training sequence is the reverse time sequence corresponding to the training video;

[0142] S940. Based on the training features of the target segmentation region and the training object labeling information, the training network is trained to obtain a segmentation feature processing network.

[0143] As an optional embodiment, a preset number of training video frame images corresponding to the training object and training object annotation information of each training video frame image are obtained. The training object annotation information may include only the training video frame image of the training object without background.

[0144] The pixels corresponding to the target object area and the pixels corresponding to the image background area in the training video frame image are identified, the target object area and the image background area are segmented, and the initial segmentation area training features corresponding to each training video frame image are obtained.

[0145] The first training sequence is the forward temporal sequence corresponding to the training videos, and the second training sequence is the reverse temporal sequence corresponding to the training videos. The network to be trained includes a correlation feature determination layer and a feature splicing layer.

[0146] Based on the first training sequence, the initial segmentation region training features corresponding to the training video frame image before each training video frame image are used as the first associated segmentation region training features. The initial segmentation region training features and the first associated segmentation region training features corresponding to each training video frame image are input into the associated feature determination layer, and the first associated training features corresponding to the previous training video frame image of each training video frame image are used as the first region classification training features corresponding to each training video frame image. Feature processing is then performed on the initial segmentation region training features corresponding to each training video frame image and the first region classification training features corresponding to each training video frame image to obtain the first associated training features corresponding to each training video frame image.

[0147] Based on the second training order, the initial segmentation region training features corresponding to the training video frame images after each training video frame image are used as the second associated segmentation region training features. The initial segmentation region training features and the second associated segmentation region training features corresponding to each training video frame image are input into the associated feature determination layer, and the second associated training features corresponding to the subsequent training video frame image of each training video frame image are used as the second region classification training features corresponding to each training video frame image. Then, feature processing is performed on the initial segmentation region training features corresponding to each training video frame image and the second region classification training features corresponding to each training video frame image to obtain the second associated training features corresponding to each training video frame image.

[0148] Input the first associated training feature and the second associated training feature into the feature splicing layer for feature splicing to obtain the target segmentation region training feature. Based on the target segmentation region training feature and the training object annotation information, train the network to be trained to obtain the segmentation feature processing network.

[0149] In the encoder-decoder structure, the training video frame images can be input into the first network to be trained for feature encoding to obtain the initial segmentation region training features corresponding to each training video frame image. Based on the first training order and the second training order, the initial segmentation region training features are input into the second network to be trained for feature decoding to obtain the target segmentation region training feature. Based on the target segmentation region training feature and the training object annotation information, train the first network to be trained and the second network to be trained to obtain the segmentation feature encoding network and the segmentation feature decoding network.

[0150] During the model training process, based on the forward time sequence order and the reverse time sequence order, the initial segmentation region training features obtained during training are processed respectively, which enables the model to learn the information in the video frame images before and after each video frame image, thereby improving the effectiveness of model training. [[ID=!1]]

[0151] As an optional embodiment, please refer to Figure 10 , training the network to be trained based on the target segmentation region training feature and the training object annotation information to obtain the segmentation feature processing network includes:

[0152] S1010. Obtain the segmentation loss information based on the target segmentation region training feature and the training object annotation information;

[0153] S1020. Train the network to be trained based on the segmentation loss information to obtain the segmentation feature processing network.

[0154] As an optional embodiment, the training object region corresponding to the training features of the target segmentation region is compared with the training object region corresponding to the training object annotation information to determine whether the background region is classified as the training object region during the region segmentation process, and segmentation region difference information is obtained. The segmentation region difference information can be pixel-level difference information. Based on the segmentation region difference information, segmentation loss information can be determined, and thus, based on the segmentation loss information, the network to be trained is trained to update the network parameters in the network to be trained, thereby obtaining a segmentation feature processing network.

[0155] During the model training process, the segmentation loss information can be determined based on pixel-level differences, and the model is trained based on the foreground-background classification error of each pixel, thereby improving the accuracy of model training.

[0156] As an optional embodiment, the image processing method can be applied to application scenarios such as video processing or online meeting background generation. By identifying clip segments of a preset video, multiple video clip segments can be obtained, and each video clip segment is a video segment between two adjacent clip points in the preset video.

[0157] Feature extraction is performed on the video frame images in each video clip segment, and the target object region and the background region in the video frame images are segmented to obtain the initial segmentation region features corresponding to each video frame image. The target object region and the background region in the video frame image can be segmented by classifying each pixel point in the video frame image. In the classification result, the feature information corresponding to the pixel points determined to be the target object region is the initial segmentation region feature.

[0158] The first preset order is the forward time sequence order corresponding to the preset video. Based on the first preset order, the initial segmentation region features corresponding to the video frame images before each video frame image are used as the first associated segmentation region features. The initial segmentation region features corresponding to each video frame image and the first associated segmentation region features corresponding to each video frame image are input into the segmentation feature processing network. Based on the first associated segmentation region features, starting from the first video frame image, a loop process is carried out in the forward order to determine the first associated feature corresponding to the first video frame image, and then the first associated feature corresponding to the first video frame image is used as the first region classification feature corresponding to the second video frame image. Based on the first region classification feature corresponding to the second video frame image and the initial segmentation region feature corresponding to the second video frame image, the first associated feature corresponding to the second video frame image is determined until the first associated feature corresponding to the penultimate video frame image is determined. The first associated feature corresponding to the penultimate video frame image is used as the first region classification feature corresponding to the last video frame image, and then feature processing is performed on the initial segmentation region feature corresponding to the last video frame image and the first region classification feature corresponding to the last video frame image to obtain the first associated feature corresponding to the last video frame image.

[0159] The second preset order is the reverse time sequence order corresponding to the preset video. Based on the second preset order, the initial segmentation region features corresponding to the video frame images after each video frame image are used as the second associated segmentation region features. The initial segmentation region features corresponding to each video frame image and the second associated segmentation region features corresponding to each video frame image are input into the segmentation feature processing network. Based on the second associated segmentation region features, starting from the last video frame image, a loop process is carried out in the reverse order to determine the second associated feature corresponding to the last video frame image, and then the second associated feature corresponding to the last video frame image is used as the second region classification feature corresponding to the penultimate video frame image. Based on the second region classification feature corresponding to the penultimate video frame image and the initial segmentation region feature corresponding to the penultimate video frame image, the second associated feature corresponding to the penultimate video frame image is determined until the second associated feature corresponding to the second video frame image is determined. Based on the second associated feature corresponding to the second video frame image, the second region classification feature corresponding to the first video frame image is determined, and then feature processing is performed on the initial segmentation region feature corresponding to the first video frame image and the second region classification feature corresponding to the first video frame image to obtain the second associated feature corresponding to the first video frame image.

[0160] In the segmentation feature processing network, feature splicing is performed on the first associated feature and the second associated feature to obtain the target segmentation region feature. Based on the target segmentation region feature, image segmentation processing is performed on each video frame image to obtain the image segmentation result corresponding to the target object.

[0161] An embodiment of the present disclosure provides an image processing method, which includes: extracting features from video frame images to obtain initial segmentation region features corresponding to each video frame image. In a segmentation feature processing network, the initial segmentation region features are processed based on a first preset order and the initial segmentation region features are processed based on a second preset order to obtain target segmentation region features. The first preset order is the forward time sequence order corresponding to a preset video, and the second preset order is the reverse time sequence order corresponding to the preset video. Image segmentation processing is performed through the target segmentation region features to obtain an image segmentation result corresponding to the target object. This method can refer to the feature recognition information corresponding to the video frame images before and after each video frame image during the region segmentation process, thereby improving the stability and accuracy of the image segmentation result.

[0162] Figure 11 is a block diagram of an image processing device shown according to an exemplary embodiment. Refer to Figure 11 , the device includes:

[0163] A video frame image acquisition module 1110, configured to acquire a preset number of video frame images corresponding to a target object;

[0164] A feature extraction module 1120, configured to extract features from video frame images to obtain initial segmentation region features corresponding to each video frame image;

[0165] A feature processing module 1130, configured to input the initial segmentation region features into a segmentation feature processing network based on a first preset order and a second preset order for feature processing to obtain target segmentation region features; the first preset order is the forward time sequence order corresponding to a preset video, and the second preset order is the reverse time sequence order corresponding to the preset video;

[0166] An image segmentation module 1140, configured to perform image segmentation processing on each video frame image based on the target segmentation region features to obtain an image segmentation result corresponding to the target object.

[0167] As an optional embodiment, the segmentation feature processing network includes an associated feature determination layer and a feature splicing layer, and the feature decoding module includes:

[0168] A first associated segmentation region feature determination unit, configured to use the initial segmentation region features corresponding to the video frame images before each video frame image as the first associated segmentation region features based on the first preset order;

[0169] The first associated feature determination unit is configured to perform inputting the initial segmentation region feature and the first associated segmentation region feature corresponding to each video frame image into an associated feature determination layer to determine associated features, so as to obtain the first associated feature corresponding to each video frame image;

[0170] The second associated segmentation region feature determination unit is configured to perform taking the initial segmentation region feature corresponding to the video frame image after each video frame image as the second associated segmentation region feature based on a second preset order;

[0171] The second associated feature determination unit is configured to perform inputting the initial segmentation region feature and the second associated segmentation region feature corresponding to each video frame image into an associated feature determination layer to determine associated features, so as to obtain the second associated feature corresponding to each video frame image;

[0172] The feature splicing unit is configured to perform inputting the first associated feature and the second associated feature into a feature splicing layer to splice features, so as to obtain the target segmentation region feature.

[0173] As an optional embodiment, the first associated feature determination unit includes:

[0174] The first region classification feature acquisition unit is configured to perform acquiring the first region classification feature corresponding to the current video frame image; the first region classification feature is obtained by inputting the first associated segmentation region feature corresponding to the current video frame image into an associated feature determination layer;

[0175] The first associated feature acquisition unit is configured to perform inputting the initial segmentation region feature and the first region classification feature corresponding to the current video frame image into an associated feature determination layer to perform feature processing, so as to obtain the first associated feature corresponding to the current video frame image;

[0176] The first video frame image update unit is configured to perform taking the first associated feature as the first region classification feature of the next video frame image of the current video frame image, and taking the next video frame image as the new current video frame image, and returning to execute the step of acquiring the first region classification feature corresponding to the current video frame image; the next video frame image is the next video frame image of the current video frame image in a first preset order.

[0177] As an optional embodiment, the second associated feature determination unit includes:

[0178] The second region classification feature acquisition unit is configured to perform acquiring the second region classification feature corresponding to the current video frame image; the second region classification feature is obtained by inputting the second associated segmentation region feature corresponding to the current video frame image into an associated feature determination layer;

[0179] The second associated feature obtaining unit is configured to perform feature processing on the initial segmentation region feature corresponding to the current video frame image and the second region classification feature corresponding to the current video frame image by inputting them into the associated feature determination layer, so as to obtain the second associated feature corresponding to the current video frame image;

[0180] The second video frame image updating unit is configured to perform using the second associated feature as the second region classification feature of the next video frame image of the current video frame image, and using the next video frame image as the new current video frame image, and return to execute the step of obtaining the second region classification feature corresponding to the current video frame image; the next video frame image is the next video frame image of the current video frame image in the second preset order.

[0181] As an optional embodiment, the video frame image obtaining module includes:

[0182] The clip segment recognition unit is configured to perform clip segment recognition on a preset video to obtain a plurality of video clip segments, and each video clip segment is a video segment between two adjacent clip points in the preset video;

[0183] The video frame image determination unit is configured to perform using the video frame images in each video clip segment as a preset number of video frame images.

[0184] As an optional embodiment, the device further includes:

[0185] The training data obtaining module is configured to perform obtaining a preset number of training video frame images corresponding to a training object and the training object annotation information of each training video frame image;

[0186] The training feature extraction module is configured to perform inputting the training video frame images into a first network to be trained for feature encoding, so as to obtain the initial segmentation region training features corresponding to each training video frame image;

[0187] The training feature processing module is configured to perform based on the first training order and the second training order, inputting the initial segmentation region training features into the network to be trained for feature processing, so as to obtain the target segmentation region training features; the first training order is the forward time sequence order corresponding to the training video, and the second training order is the reverse time sequence order corresponding to the training video;

[0188] The network training module is configured to perform training the network to be trained based on the target segmentation region training features and the training object annotation information, so as to obtain the segmentation feature processing network.

[0189] As an optional embodiment, the network training module includes:

[0190] A segmentation loss determination unit, configured to obtain segmentation loss information based on training features of a target segmentation region and training object annotation information;

[0191] A network training unit, configured to train a network to be trained based on the segmentation loss information to obtain a segmentation feature processing network.

[0192] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0193] Figure 12 is a block diagram of an electronic device for image processing shown according to an exemplary embodiment. The electronic device may be a server, and its internal structural diagram may be as Figure 12 shown. The electronic device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements an image processing method.

[0194] Those skilled in the art can understand that Figure 12 the structure shown in is only a block diagram of some structures related to the solution of the present disclosure, and does not constitute a limitation on the electronic device to which the solution of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0195] In an exemplary embodiment, an electronic device is further provided, including: a processor; a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the instructions to implement the image processing method in the embodiments of the present disclosure.

[0196] In an exemplary embodiment, a computer-readable storage medium including instructions is further provided, such as a memory 1204 including instructions. The above instructions can be executed by a processor 1220 of an electronic device 1200 to complete the above method. Optionally, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0197] In an exemplary embodiment, a computer program product including instructions is further provided. When it runs on a computer, it causes the computer to execute the image processing method in the embodiments of the present disclosure.

[0198] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. This computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in this application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0199] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application aims to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include well-known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0200] It should be understood that the present disclosure is not limited to the exact structures already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a preset number of video frame images corresponding to the target object; Extracting features from the video frame images to obtain initial segmentation region features corresponding to each of the video frame images; Based on a first preset order, taking the initial segmentation region features corresponding to the video frame image before each of the video frame images as the first associated segmentation region features; Inputting the initial segmentation region feature and the first associated segmentation region feature corresponding to each of the video frame images into an associated feature determination layer in a segmentation feature processing network for associated feature determination, thereby obtaining a first associated feature corresponding to each of the video frame images; Based on a second preset order, taking the initial segmentation region features corresponding to the video frame images following each of the video frame images as second associated segmentation region features; Inputting the initial segmented region feature and the second associated segmented region feature corresponding to each of the video frame images into the associated feature determination layer to perform associated feature determination, thereby obtaining the second associated feature corresponding to each of the video frame images; Inputting the first associated features and the second associated features into a feature splicing layer in a segmentation feature processing network for feature splicing to obtain target segmentation region features, wherein the first preset order is a positive temporal order corresponding to a preset video, and the second preset order is a reverse temporal order corresponding to the preset video; Based on the target segmentation region features, image segmentation processing is performed on each of the video frame images to obtain an image segmentation result corresponding to the target object.

2. The image processing method according to claim 1, wherein: Inputting the initial segmented region feature and the first associated segmented region feature corresponding to each of the video frame images into the associated feature determination layer to perform associated feature determination, and obtaining the first associated feature corresponding to each of the video frame images includes: Obtaining a first region classification feature corresponding to the current video frame image; the first region classification feature is obtained by inputting a first associated segmentation region feature corresponding to the current video frame image into the associated feature determination layer; Inputting the initial segmentation region feature corresponding to the current video frame image and the first region classification feature corresponding to the current video frame image into the associated feature determination layer for feature processing to obtain the first associated feature corresponding to the current video frame image; Use the first associated feature as the first region classification feature of the next video frame image of the current video frame image, and use the next video frame image as the new current video frame image, and return to execute the step of obtaining the first region classification feature corresponding to the current video frame image; the next video frame image is the next video frame image of the current video frame image under the first preset order.

3. The image processing method according to claim 1, wherein: Inputting the initial segmented region feature and the second associated segmented region feature corresponding to each of the video frame images into the associated feature determination layer to perform associated feature determination, and obtaining the second associated feature corresponding to each of the video frame images includes: Obtaining a second region classification feature corresponding to the current video frame image; the second region classification feature is obtained by inputting a second associated segmentation region feature corresponding to the current video frame image into the associated feature determination layer; Inputting the initial segmentation region feature corresponding to the current video frame image and the second region classification feature corresponding to the current video frame image into the associated feature determination layer for feature processing to obtain the second associated feature corresponding to the current video frame image; Use the second associated feature as the second region classification feature of the next video frame image of the current video frame image, and use the next video frame image as the new current video frame image, and return to execute the step of obtaining the second region classification feature corresponding to the current video frame image; the next video frame image is the next video frame image of the current video frame image under the second preset order.

4. The image processing method according to claim 1, wherein: The acquiring of a preset number of video frame images corresponding to the target object includes: Identifying clips of a preset video to obtain a plurality of video clips, each of which is a video clip between two adjacent clipping points in the preset video; The video frame images in each of the video clips are used as the preset number of video frame images.

5. The image processing method according to claim 1, wherein: The method further comprises: Acquire a preset number of training video frame images corresponding to the training object and training object annotation information of each of the training video frame images; Performing feature extraction on the training video frame images to obtain initial segmentation region training features corresponding to each training video frame image; Based on a first training sequence and a second training sequence, the training features of the initial segmented region are input into the to-be-trained network for feature processing to obtain training features of the target segmented region; the first training sequence is a positive temporal sequence corresponding to the training video, and the second training sequence is a reverse temporal sequence corresponding to the training video; Based on the target segmentation region training features and the training object labeling information, the network to be trained is trained to obtain the segmentation feature processing network.

6. The image processing method according to claim 5, characterized in that The training of the network to be trained based on the target segmentation region training features and the training object labeling information to obtain the segmentation feature processing network includes: Obtaining segmentation loss information based on the target segmentation region training features and the training object labeling information; Based on the segmentation loss information, the network to be trained is trained to obtain the segmentation feature processing network.

7. An image processing device, characterized in that The device comprises: A video frame image acquisition module is configured to acquire a preset number of video frame images corresponding to the target object; A feature extraction module is configured to perform feature extraction on the video frame image to obtain initial segmentation region features corresponding to each video frame image; A first associated segmented region feature determination module is configured to execute, based on a first preset order, taking the initial segmented region feature corresponding to the video frame image before each of the video frame images as the first associated segmented region feature; A first correlation feature determination module is configured to input the initial segmentation region feature and the first correlation segmentation region feature corresponding to each of the video frame images into a correlation feature determination layer in the segmentation feature processing network to perform correlation feature determination, thereby obtaining a first correlation feature corresponding to each of the video frame images; The second associated segmentation region feature is configured to execute, based on a second preset order, the initial segmentation region feature corresponding to each video frame image after the video frame image as the second associated segmentation region feature; A second correlation feature determination module is configured to input the initial segmented region feature and the second correlation segmented region feature corresponding to each of the video frame images into the correlation feature determination layer to perform correlation feature determination, thereby obtaining a second correlation feature corresponding to each of the video frame images; a feature splicing module configured to execute feature splicing by inputting the first associated features and the second associated features into a feature splicing layer in a segmentation feature processing network to obtain target segmentation region features, wherein the first preset order is a forward temporal order corresponding to the preset video, and the second preset order is a reverse temporal order corresponding to the preset video; The image segmentation module is configured to perform image segmentation processing on each of the video frame images based on the target segmentation area features to obtain an image segmentation result corresponding to the target object.

8. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image processing method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 6 is implemented.

Images