Image processing method, device and storage medium

Abstract

The embodiments of the present application provide an image processing method, device and storage medium. In the image processing method provided in the embodiments of the present application, the image to be processed is segmented to obtain a first estimated area corresponding to the first target image and a second estimated area corresponding to the second target image. According to the contour features of the two estimated image areas, the target area on the image to be processed that contains the boundary of the first target image and the second target image can be determined; then, the target area on the image to be processed is segmented to correct the first estimated area and the second estimated area. Based on the above-mentioned image processing process, after the segmentation result of the image to be processed is obtained, the segmentation result of the image to be processed is corrected according to the segmentation result of the target area on the image to be processed that is not easy to accurately segment, thereby improving the segmentation accuracy of the image and effectively increasing the availability and reliability of the image segmentation result.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device and storage medium. Background Art

[0002] In some image processing application scenarios, there is a need for image segmentation, which usually indicates segmenting a certain image region in an image from the entire image, so as to perform other image processing operations based on the segmented image region.

[0003] However, the existing image segmentation technology has limited segmentation accuracy and cannot accurately segment images when there are shadows or other image interferences, which is not conducive to subsequent image processing operations. Therefore, a new solution is needed. Summary of the invention

[0004] Multiple aspects of the present application provide an image processing method, device and storage medium for improving the segmentation accuracy of an image during image segmentation.

[0005] An embodiment of the present application provides an image processing method, including: segmenting an image to be processed to obtain an estimated subject area corresponding to a subject on the image to be processed and an estimated shadow area corresponding to the shadow of the subject; determining a target area on the image to be processed that includes a boundary between the subject and the shadow of the subject based on contour features of the estimated subject area and the estimated shadow area; and re-segmenting the target area to obtain the subject and the shadow of the subject on the image to be processed.

[0006] An embodiment of the present application provides an image processing method, including: segmenting an image to be processed to obtain a first estimated area corresponding to a first target image and a second estimated area corresponding to a second target image; determining a target area on the image to be processed that includes a boundary between the first target image and the second target image based on contour features of the first estimated area and the second estimated area; and re-segmenting the target area on the image to be processed to correct the first estimated area and the second estimated area.

[0007] An embodiment of the present application also provides an image processing device, comprising: a memory and a processor; wherein the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to: segment the image to be processed to obtain a first estimated area corresponding to the first target image and a second estimated area corresponding to the second target image; determine the target area on the image to be processed based on contour features of the first estimated area and the second estimated area; and re-segment the target area on the image to be processed to correct the first estimated area and the second estimated area.

[0008] The embodiment of the present application also includes a computer-readable storage medium storing a computer program, which, when executed, can implement the image processing method provided in the embodiment of the present application.

[0009] In the image processing method provided in the embodiment of the present application, the image to be processed is segmented to obtain a first estimated area corresponding to the first target image and a second estimated area corresponding to the second target image. According to the contour features of the two estimated image areas, the target area on the image to be processed that contains the boundary of the first target image and the second target image can be determined; then, the target area on the image to be processed is segmented to correct the first estimated area and the second estimated area. Based on the above image processing process, after the segmentation result of the image to be processed is obtained, the segmentation result of the image to be processed is corrected according to the segmentation result of the target area on the image to be processed that is not easy to accurately segment, thereby improving the segmentation accuracy of the image and effectively increasing the availability and reliability of the image segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0011] Figure 1 A flowchart of an image processing method provided by an exemplary embodiment of the present application;

[0012] Figure 2a A flowchart of an image processing method provided for another exemplary embodiment of the present application;

[0013] Figure 2b An illustration of obtaining a target area on an image to be processed provided by an exemplary embodiment of the present application;

[0014] Figure 2c An illustration of a first image mask provided by an exemplary embodiment of the present application;

[0015] Figure 2d An illustration of a first image mask provided for another exemplary embodiment of the present application;

[0016] Figure 2e An illustration of correcting the first estimated area and the second estimated area provided by an exemplary embodiment of the present application;

[0017] Figure 2f A flowchart of an image processing method provided for another exemplary embodiment of the present application;

[0018] Figure 3aA schematic flow chart of a training method for a second image segmentation model provided in an exemplary embodiment of the present application;

[0019] Figure 3b A schematic diagram of a model training process provided for an exemplary application scenario example of the present application;

[0020] Figure 4 is a structural schematic diagram of an image processing device provided by an exemplary embodiment of the present application;

[0021] Figure 5 It is a structural schematic diagram of an image processing device provided by another exemplary embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0023] The image segmentation operation refers to segmenting a certain image area from the entire image to perform other image processing operations based on the segmented image area. For example, segmenting the foreground person in the original image from the original background image, replacing the original background image with a new background image, to achieve the replacement of the person background. However, the segmentation accuracy of existing image segmentation technologies is limited, and it is not possible to accurately segment the image when there is a shadow in the image, which is not conducive to subsequent image processing operations. For example, when there is a shadow on the foreground object on the image, the result of segmenting the foreground object and the background in the image usually contains shadow information, resulting in poor usability of the segmentation result.

[0024] In view of the above technical problems, a solution is provided in some embodiments of the present application. The technical solutions provided by the embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0025] Figure 1 A flowchart of an image processing method provided by an exemplary embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0026] Step 101 : Segment the image to be processed to obtain a first estimated region corresponding to a first target image and a second estimated region corresponding to a second target image.

[0027] Step 102: Determine a target region on the image to be processed that includes a boundary between the first target image and the second target image according to contour features of the first estimated region and the second estimated region.

[0028] Step 103: re-segment the target area on the image to be processed to correct the first estimated area and the second estimated area.

[0029] In this embodiment, the image to be processed may be an image input by a user, or an image transmitted by other devices or applications. The image to be processed includes a first target image and a second target image. The first target image and the second target image are usually in contact or close to each other, and the colors of the two are similar, so that it is not easy to separate the first target image and the second target image. For example, in some embodiments, the first target image may be implemented as a foreground object on the image to be processed, and the second target image may be implemented as a shadow, noise, etc. near the foreground object, which is not limited in this embodiment.

[0030] In this embodiment, for the convenience of description, the first estimated area and the second estimated area are used to mark the segmentation result obtained by segmenting the image to be processed. Usually, the accuracy of the segmentation result does not meet the specified requirements. Among them, most of the pixels in the first estimated area belong to the first target image on the image to be processed, but some pixels may belong to the second target image on the image to be processed; most of the pixels in the second estimated area belong to the second target image on the image to be processed, but some pixels may belong to the first target image on the image to be processed.

[0031] In this embodiment, the boundary of the first target image and the second target image refers to the real boundary of the first target image and the second target image. Due to the limitation of segmentation accuracy, the real boundary does not coincide with the boundary of the first estimated region and the second estimated region, but may be distributed at a position associated with the contours of the first estimated region and the second estimated region.

[0032] The target area on the image to be processed refers to an area on the image to be processed that may contain the boundary between the first target image and the second target image. Re-segmenting the target area on the image to be processed can focus the segmentation operation on the boundary between the first target image and the second target image, which is conducive to improving the segmentation accuracy. After re-segmenting the target area on the image to be processed, the first estimated area and the second estimated area can be corrected according to the result of the re-segmentation, thereby obtaining the first target image and the second target image with higher segmentation accuracy.

[0033] In this embodiment, the image to be processed is segmented to obtain a first estimated area corresponding to the first target image and a second estimated area corresponding to the second target image. According to the contour features of the two estimated image areas, the target area on the image to be processed that includes the boundary of the first target image and the second target image can be determined; then, the target area on the image to be processed is segmented to correct the first estimated area and the second estimated area. Based on the above image processing process, after obtaining the segmentation result of the image to be processed, the segmentation result of the image to be processed is corrected according to the segmentation result of the target area on the image to be processed that is difficult to accurately segment, thereby improving the segmentation accuracy of the image and effectively increasing the availability and reliability of the image segmentation result.

[0034] In the above and below embodiments of the present application, an optional implementation of re-segmenting the target area on the image to be processed to correct the first estimated area and the second estimated area is: identifying some pixels belonging to the first target image and some pixels belonging to the second target image in the target area on the image to be processed to re-segment the target area on the image to be processed; then, correcting the first estimated area and the second estimated area according to the result of the re-segmentation to obtain the first target image and the second target image.

[0035] In some exemplary embodiments, one way to correct the first estimated region and the second estimated region according to the result of the re-segmentation can be implemented as follows: according to the result of the re-segmentation, the contour of the junction of the first estimated region and the second estimated region is adjusted to correct the distribution position of the boundary between the first target image and the second target image. According to the corrected boundary, the first target image and the second target image with higher segmentation accuracy can be obtained.

[0036] In other exemplary embodiments, a method of correcting the first estimated area and the second estimated area according to the result of re-segmentation can be implemented as follows: using the image area obtained by re-segmenting the target area on the image to be processed to replace the image area corresponding to the target area on the first estimated area and the second estimated area.

[0037] It should be understood that the above two optional correction methods are used to exemplify the embodiments of the present application, and it is not excluded that in other optional embodiments of the present application, correction methods different from the above two implementation methods are used to correct the first estimated area and the second estimated area.

[0038] In the above and following embodiments of the present application, when segmenting the image to be processed and re-segmenting the target area on the image to be processed, at least one of the following image segmentation methods may be used: a segmentation method based on a threshold, a segmentation method based on a region, a segmentation method based on an edge, a segmentation method based on cluster analysis theory, a segmentation method based on fuzzy set theory, a segmentation method based on gene coding theory, a segmentation method based on wavelet transform theory, and a segmentation method based on a neural network model (NN), which is not limited in the present application. In the following embodiments, a segmentation method based on a neural network model is used as an example for exemplary description.

[0039] Figure 2a A flowchart of an image processing method provided by another exemplary embodiment of the present application is shown in FIG. Figure 2a As shown, the method includes:

[0040] Step 201a, inputting the image to be processed into a first image segmentation model; the first image segmentation model uses sample images including the first target image and the second target image as training samples, and is trained by minimizing the loss function of the first target image and / or the second target image.

[0041] Step 202a: Run the first image segmentation model to identify a first estimated region corresponding to the first target image and a second estimated region corresponding to the second target image from the image to be processed.

[0042] Step 203a: Expand the outline of the first estimated region to obtain a first expanded outline, and expand the outline of the second estimated region to obtain a second expanded outline.

[0043] Step 204a: Acquire the overlapping area of ​​the first extended contour and the second extended contour, and use the image area on the image to be processed corresponding to the overlapping area as the target area on the image to be processed.

[0044] Step 205a, marking the target area on the image to be processed, and inputting the marked image to be processed into a second image segmentation model; the second image segmentation model uses sample images marked with the first target image and the second target image and marked with the target area as training samples, and is trained by minimizing the loss function of the target area.

[0045] Step 206a: Run the second image segmentation model to identify some pixels belonging to the first target image and some pixels belonging to the second target image from the target area on the image to be processed.

[0046] Step 207a, replacing the first estimated area and the image area on the second estimated area corresponding to the target area on the image to be processed with some pixels belonging to the first target image and some pixels belonging to the second target image, to obtain the first target image and the second target image.

[0047] In step 201a, the first image segmentation model may be implemented based on a neural network model.

[0048] When training the first image segmentation model, a sample image including the first target image and the second target image may be obtained as a training sample, the first target image and the second target image on the training sample are labeled, and the labeled training sample is input into the neural network model.

[0049] In the neural network model, the image features of the training sample are extracted through the convolution kernel provided by the convolution layer, and the first target image and the second target image on the training sample are recognized based on the image features. After obtaining the recognition result, the difference between the recognition result and the first target image marked on the training sample is obtained, and the loss function of the first target image is determined according to the difference, and it is judged whether the loss function is less than the set threshold. If the judgment is no, the model parameters of the neural network model are updated, and the above training process is repeated until the loss function of the first target image is less than the set threshold.

[0050] Alternatively, after obtaining the recognition result, the difference between the recognition result and the second target image marked on the training sample is obtained, and the loss function of the second target image is determined according to the difference, and it is determined whether the loss function is less than a set threshold. If the judgment is no, the model parameters of the neural network model are updated, and the above training process is repeated until the loss function of the second target image is less than the set threshold.

[0051] Alternatively, after obtaining the recognition result, the loss function of the first target image and the loss function of the second target image are obtained respectively, and it is determined whether the two loss functions are both less than a set threshold value. If the judgment is no, the model parameters of the neural network model are updated, and the above training process is repeated until the loss function of the first target image and the loss function of the second target image are both less than the set threshold value.

[0052] Optionally, in this embodiment, the loss function may be implemented as a cross entropy loss function, and this embodiment includes but is not limited to this.

[0053] In step 202a, when running the first image segmentation model, the first image segmentation model can extract image features of the image to be processed based on the model parameters obtained by pre-training, and identify the image area on the image to be processed that may contain the first target image as the first estimated area based on the extracted image features, and identify the image area on the image to be processed that may contain the second target image as the second estimated area.

[0054] In step 203a, after obtaining the first estimated area and the second estimated area, the outline of the first estimated area can be extended to obtain a first extended outline, and the outline of the second estimated area can be extended to obtain a second extended outline, such as Figure 2b shown.

[0055] Optionally, before performing the contour extension, contour detection may be performed on the first estimated region and the second estimated region to identify the contour of the first estimated region and the contour of the second estimated region.

[0056] Optionally, in some embodiments, an edge detection operator, such as a Laplace operator, a Sobel operator, a Canny operator, etc., may be used to detect the contour of the first estimated area and the contour of the second estimated area. Optionally, in other embodiments, a method based on a human visual perception model may be used to detect the contour of the first estimated area and the contour of the second estimated area. It should be understood that the above two methods are only used for exemplary description, and this embodiment includes but is not limited to the above two methods.

[0057] In some exemplary embodiments, when the contours of the first estimated region and the second estimated region are extended, a morphological algorithm based on erosion and dilation processing may be used.

[0058] When performing erosion and dilation, a kernel of any shape can be slid on the image. For erosion, the kernel extracts the minimum pixel value in the kernel coverage area each time it slides, and replaces the pixel value of the pixel corresponding to the anchor point in the kernel (usually the center point in the kernel) with the minimum pixel value. For dilation, the kernel extracts the maximum pixel value in the kernel coverage area each time it slides, and replaces the pixel value of the pixel corresponding to the anchor point in the kernel with the maximum pixel value. In other words, the erosion operation can widen the area with smaller grayscale values ​​on the image, and the dilation operation can widen the area with larger grayscale values ​​on the image.

[0059] Based on this, when the grayscale values ​​of the pixels on the contour of the first estimated area and the contour of the second estimated area are small (for example, less than the set grayscale threshold), the contour of the first estimated area and the contour of the second estimated area can be eroded to obtain the first extended contour and the second extended contour respectively. The radius of the erosion process can be set according to actual needs, and this embodiment does not limit this.

[0060] When the grayscale values ​​of the pixels on the contour of the first estimated area and the contour of the second estimated area are large (for example, greater than or equal to the set grayscale threshold), the contour of the first estimated area and the contour of the second estimated area may be expanded to obtain a first extended contour and a second extended contour, respectively. The radius of the expansion process may be set according to actual needs, and this embodiment does not limit this.

[0061] In other exemplary embodiments, when the outline of the first estimated area and the second estimated area is extended, a method based on brush strokes can be used. Optionally, in this embodiment, when the outline of the first estimated area is extended, the grayscale value of the brush can be set to a value close to the grayscale value of the outline of the first estimated area, and the set brush width is used to draw along the outline of the first estimated area to widen the outline of the first estimated area to obtain a first extended outline. Similarly, when the outline of the second estimated area is extended, the grayscale value of the brush can be set to a value close to the grayscale value of the outline of the second estimated area, and the set brush width is used to draw along the outline of the second estimated area to widen the outline of the second estimated area to obtain a second extended outline. Among them, the brush widths used for extending the outlines of the first estimated area and the second estimated area can be the same or different, and this embodiment does not limit it.

[0062] Of course, the contour extension methods listed in the above two implementations are only used to exemplify the implementation of this step. In other optional embodiments, other contour extension methods are not excluded to extend the contours of the first estimated area and the second estimated area, which will not be repeated in this embodiment.

[0063] In step 204a, after obtaining the first extended contour and the second extended contour, the overlapping area of ​​the first extended contour and the second extended contour can be obtained. Then, the image area on the image to be processed corresponding to the overlapping area is used as the target area on the image to be processed, such as Figure 2b It should be understood that the boundary between the first target image and the second target image may be located within the target area on the image to be processed.

[0064] In step 205a, optionally, the purpose of marking the target area on the image to be processed is to instruct the second image segmentation model to focus the image segmentation processing on the target area on the image to be processed. The following will be exemplified in conjunction with some optional implementations.

[0065] In some exemplary embodiments, the pixel coordinates corresponding to the target area on the image to be processed may be obtained. Then, the pixel coordinates are marked on the image to be processed, and the marked image to be processed is input into the second image segmentation model. Furthermore, the second image segmentation model may determine the calculation range of the loss function based on the position mark of the pixel coordinates.

[0066] In other exemplary embodiments, a first image mask having the same resolution as the image to be processed may be generated. When generating the first image mask, the pixels on the first image mask may be divided into first pixels, second pixels, and third pixels. The first pixel refers to a pixel on the first image mask corresponding to the target area on the image to be processed; the second pixel refers to a pixel on the first image mask corresponding to the first estimated area except the first pixel; the third pixel refers to other pixels on the first image mask except the first pixel and the second pixel.

[0067] In this embodiment, optionally, the gray value of the first pixel can be set to a set mark value, and the gray values ​​of the second pixel and the third pixel can be set to non-mark values. When the second image segmentation model performs the segmentation operation, it can identify which image areas of the pixels on the image to be processed are the image areas to be segmented, and which pixels can participate in the calculation of the loss function according to the mark value. The specific values ​​of the mark value and the non-mark value can be set according to actual needs, and this embodiment does not limit it.

[0068] Optionally, in some embodiments, the mark value may be set to 128. When the grayscale value of the first pixel is 128, the grayscale value of the second pixel may be set to 255, and the grayscale value corresponding to the third pixel may be set to 0, so that the human eye has a better visual effect, such as Figure 2c Based on the first image mask obtained by the above parameters, the second image segmentation model can identify the pixels corresponding to 128 on the image to be processed during the image segmentation process, and focus the segmentation on these pixels to improve the segmentation accuracy.

[0069] Optionally, in some other embodiments, the mark value may be set to 1, and the non-mark value may be set to 0. Thus, a binary first image mask may be obtained in which the gray value of the first pixel is 1, and the gray values ​​of the second pixel and the third pixel are 0, such as Figure 2dThe first image mask obtained based on the above parameters can assist the second image segmentation model in performing filtering operations on the image to be processed during the image segmentation process, filtering out other image regions except the target region, so as to eliminate the interference of other image regions except the target region on the image to be processed on the segmentation process and improve the segmentation accuracy.

[0070] The second image segmentation model can be obtained by training based on a neural network model. During training, a sample image labeled with the first target image and the second target image and marked with a target area can be obtained as a training sample, and trained by minimizing the loss function of the target area. This part of the training process will be described in subsequent embodiments and will not be repeated here.

[0071] It should be understood that the first image mask is a single-channel image, and the image to be processed is usually a three-channel image (for example, an RGB image with three channels of red, green, and blue), so the first image mask cannot directly perform a marking operation on the image to be processed. To solve this problem, the embodiment of the present application adds a convolution processing module in the neural network model for performing convolution processing on the first image mask. The convolution processing module can be set at the input end of the first image mask in the neural network model.

[0072] Based on this, in step 206a, when running the second image segmentation model, optionally, within the second image segmentation model, a single-channel first convolution kernel provided by the mask convolution module can be used to perform convolution processing on the first image mask to obtain a first feature map, and in the first convolution layer, a three-channel second convolution kernel can be used to perform convolution processing on the image to be processed to obtain a second feature map.

[0073] In this embodiment, the first convolution kernel and the second convolution kernel are set to be the same in number and size. Further, after the first image mask is convolved by the first convolution kernel, a first feature map having the same number of channels as the first convolution kernel can be obtained. After the image to be processed is convolved by the second convolution kernel, a second feature map having the same number of channels as the second convolution kernel can be obtained.

[0074] For example, the number of the first convolution kernel and the number of the second convolution kernel are both N. After the first image mask is convolved by the mask convolution module, an N-channel first feature map can be obtained; after the image to be processed is convolved by the first convolution layer, an N-channel second feature map can be obtained.

[0075] Since the resolution of the image to be processed is the same as that of the first image mask, the resolution of the first feature map and the second feature map are also the same. Based on the above, when the resolution of the first feature map and the second feature map are the same and the number of channels is the same, the first feature map and the second feature map can be bitwise superimposed to obtain a third feature map.

[0076] In this step, after the first convolution layer is completed, the specific information on the second feature map is filtered out according to the gray value distribution characteristics of the first feature map. On the one hand, the calculation focus of other calculation layers can be concentrated on the target area of ​​the image to be processed. On the other hand, the calculation amount of other calculation layers after the first convolution layer is effectively reduced, saving computing resources.

[0077] It should be understood that in other optional implementations, the first feature map may be bitwise superimposed with the output results of the second, third or other convolutional layers, and this embodiment does not impose any limitation thereto.

[0078] Among them, other computing layers after the convolution layer may include other convolution layers, rectification layers, pooling layers, fully connected layers, etc. in the neural network model except the first convolution layer, which will not be described in detail. It should be understood that after the calculation of the other computing layers, the neural network model can output the recognition result.

[0079] Optionally, one form of expression of the output result of the neural network model is: identifying some pixels belonging to the first target image and some pixels belonging to the second target image in the target area on the image to be processed.

[0080] Next, in step 207a, optionally, the image area corresponding to the target area on the image to be processed in the first estimated area and the second estimated area may be replaced with some pixels belonging to the first target image and some pixels belonging to the second target image to obtain the first target image and the second target image. Specific examples will be used for exemplary description below.

[0081] In some exemplary embodiments, Figure 2e As shown, assuming that in the first estimated area, the pixel corresponding to the target area on the image to be processed is pixel A, and the pixel corresponding to the target area on the image to be processed is pixel B in the second estimated area; assuming that part of the pixels belonging to the first target image in the identified target area on the image to be processed is pixel A', and part of the pixels belonging to the second target image is pixel B'; then, when correcting the first estimated area and the second estimated area according to the result of re-segmentation, pixel A' can be used to replace pixel A, and pixel B' can be used to replace pixel B, so as to finally obtain the first target image and the second target image with higher segmentation accuracy.

[0082] In this embodiment, the image to be processed is segmented to obtain a first estimated area corresponding to the first target image and a second estimated area corresponding to the second target image. According to the contour features of the two estimated image areas, the target area on the image to be processed that includes the boundary of the first target image and the second target image can be determined; then, the target area on the image to be processed is segmented to correct the first estimated area and the second estimated area. Based on the above image processing process, after obtaining the segmentation result of the image to be processed, the segmentation result of the image to be processed is corrected according to the segmentation result of the target area on the image to be processed that is not easy to accurately segment, thereby improving the segmentation accuracy of the image and effectively increasing the availability and reliability of the image segmentation result.

[0083] Figure 2f A flowchart of an image processing method provided by another exemplary embodiment of the present application is shown in FIG. Figure 2f As shown, the method includes:

[0084] Step 201f, segmenting the image to be processed to obtain an estimated main body region corresponding to the main object on the image to be processed and an estimated shadow region corresponding to the shadow of the main object.

[0085] Step 202f: Determine a target area on the image to be processed that includes a boundary between the main object and the shadow of the main object based on the contour features of the estimated main object area and the estimated shadow area.

[0086] Step 203f: re-segment the target area to obtain the main object and the shadow of the main object on the image to be processed.

[0087] In this embodiment, the main object on the image to be processed may be a foreground object on the image to be processed or a main object of interest to human vision. For example, when the image to be processed is implemented as a product poster, the main object may be a product on the poster, such as shoes, clothes, fruits, electronic products, etc. For another example, when the image to be processed is implemented as a landscape image, the main object may be a person, building, vehicle, etc. on the image. The shadow of the main object refers to the shadow caused by uneven shooting light, or the shadow distributed near the main object for aesthetic design.

[0088] Optionally, the optional implementation method of obtaining the target area can refer to the description of the above embodiment, which is not repeated here. After obtaining the target area, some pixels belonging to the estimated main area and some pixels belonging to the estimated shadow area in the target area can be identified to re-segment the main object on the image to be processed; then, the estimated main area and the estimated shadow area are corrected according to the result of the re-segmentation to obtain the main object and the shadow of the main object on the image to be processed.

[0089] Based on the technical solution of this embodiment, the main object and its shadow on the image to be processed can be segmented for the first time to obtain the estimated main area and the estimated shadow area. Then, according to the contour features of the estimated main area and the estimated shadow area, the target area including the boundary between the main object and its shadow is determined from the image to be processed; then, the target area on the image to be processed is segmented again to correct the result of the first segmentation, so that the main object and its shadow obtained by the final segmentation have higher accuracy, effectively increasing the availability and reliability of the image segmentation result.

[0090] Figure 3a A flowchart of a training method for a second image segmentation model provided by an exemplary embodiment of the present application is shown in FIG. Figure 3a As shown, the method includes:

[0091] Step 301: Acquire a sample image annotated with a first target image and a second target image, and generate a second image mask having the same resolution as the sample image according to a target area on the sample image.

[0092] Step 302: Input the sample image and the second image mask into the neural network model.

[0093] Step 303: Use the single-channel first convolution kernel provided by the mask convolution module to convolve the second image mask to obtain a first feature sample map, and use the three-channel second convolution kernel to convolve the sample image in the first convolution layer to obtain a second feature sample map; the first convolution kernel and the second convolution kernel have the same size and number.

[0094] Step 304: superimpose the first feature sample image and the second feature sample image bit by bit to obtain a third feature sample image, and input the third feature sample image into other computing layers after the first convolutional layer.

[0095] Step 305: Obtain the recognition result of the target area on the sample image by the other computing layer according to the third feature sample image, and calculate the loss function of the target area.

[0096] Step 306 , determine whether the loss function of the target area is less than a set threshold; if yes, execute step 307 ; if no, execute step 308 .

[0097] Step 307: Determine model parameters of the second image segmentation model according to the parameters of the first convolutional layer and other computing layers.

[0098] Step 308: update the parameters of the mask convolution module, the first convolution layer and the other computing layers, and re-execute step 303.

[0099] In this embodiment, the sample image is marked with a first target image and a second target image, and a target area is marked on the sample image. Optionally, the solution described in the above embodiment can be used to segment the sample image using the first image segmentation model, and the solution described in the above embodiment can be used to obtain the target area on the sample image based on the segmentation result of the sample image, which will not be described in detail.

[0100] The second image mask is used to mark the target area on the sample image during the training phase of the model to reduce the influence of interference factors on the sample image on the training process. The method for setting the pixel grayscale value in the second image mask can refer to the relevant records of the first image mask in the above embodiment, and will not be repeated here.

[0101] In this embodiment, due to the existence of the second image mask, other computing layers in the neural network model can adjust the pixel distribution range involved in the loss function calculation to the range where the target area is located in the process of identifying the target area on the sample image according to the third feature sample map. Then, the second image segmentation model obtained by continuous iterative training can focus on segmenting the target area to improve the segmentation accuracy.

[0102] The following sections will combine Figure 3b , using a specific application scenario to further illustrate the technical solution of the embodiment of the present application.

[0103] In some typical application scenarios, when the object in the image is a black shoe and the black shoe has a black shadow, it is difficult to separate the shoe and the shadow with high accuracy. Based on the records of the above embodiments, the following operations can be performed to solve the above technical problems:

[0104] First, a sample image containing shoes and shadows may be obtained, and an image semantic segmentation model may be trained based on the sample image and the neural network model. In this embodiment, two image semantic segmentation models may be obtained through two training phases.

[0105] In the first training stage, the shoe outline can be outlined along the edge of the shoe on the sample image, and the shadow outline can be outlined along the edge of the shadow area to obtain a labeled sample image. Then, the first image semantic segmentation model can be trained based on the two categories of shoes and shadows. During the training process, the loss function can be calculated based on the entire sample image until the trained image semantic segmentation model can better segment the shoe and shadow area.

[0106] However, usually, there is no strict boundary between the shadow and the background area. Therefore, when annotating samples, it is more difficult to mark the boundary of the shadow in the shadow gradient area. Based on the above reasons, the actual marked shadow may have a certain gap with the distribution position of the real shadow. Therefore, in the process of training the first image semantic segmentation model, when calculating the loss function of the whole image, the gap caused by the shadow annotation will cause the reliability of the loss function to deteriorate. As a result, the first image segmentation model cannot accurately segment the shoes and shadows. The segmentation error is particularly obvious when the color of the shadow is very similar to the color of the shoe or the shape of the bottom of the shoe is uneven.

[0107] Therefore, in this embodiment, a re-segmentation solution is proposed, and in the process of training the second image semantic segmentation model used for re-segmentation, the calculation process of the loss function is concentrated on the sole area where the shadow and the shoe meet. This method can focus the optimization direction more on the boundary between the sole and the shadow, which is difficult to segment accurately, and effectively avoid the impact of incorrect segmentation of the boundary between the shadow and the background area on the reliability of the loss function. The following will explain in detail.

[0108] In the second training stage, the segmentation result of the first image semantic segmentation model can be used as the training basis. For the convenience of description, the area where the shoe is located obtained by the first image semantic segmentation model is marked as the first area, and the area where the shadow is located is marked as the second area.

[0109] Next, the contours of the first region and the second region may be detected. For example, each pixel on the contours of the first region and the second region may be eroded with a certain radius, or the contours of the first region and the second region may be widened with a brush of a certain width.

[0110] After the extended contours of the first region and the second region are obtained, the overlapping region of the two extended contours can be used as the region where the boundary between the shoe and the shadow may exist. For the convenience of description, the region where the boundary between the shoe and the shadow may exist is marked as the target region.

[0111] Next, an image mask with the same resolution as the sample image is generated, the grayscale values ​​of the pixels corresponding to the target area on the image mask are set to 128, the grayscale values ​​of the remaining pixels in the first area are set to 255, and the grayscale values ​​of the remaining pixels in the second area are set to 0.

[0112] Next, the sample image and image mask are input into the neural network model to train the second image semantic segmentation model. The second image semantic segmentation model uses the model structure of the first image semantic segmentation model and adds a convolution processing module to the input end corresponding to the image mask. The convolution processing module provides the same number of convolution kernels as the first convolution layer.

[0113] For example, the sample image is a 3-channel image with a resolution of 800*800. Assume that the first convolution layer contains 64 convolution kernels, and the size of each convolution kernel is 3c*3h*3w (where c is the number of channels, h is the number of pixels in the vertical direction of the convolution kernel, and w is the number of pixels in the horizontal direction of the convolution kernel). After the sample image is calculated by the first convolution layer, a 64-channel feature map with a resolution of 800*800 can be obtained.

[0114] In order to make the image mask work on the sample image, 64 1c*3h*3w convolution kernels can be set in the convolution processing module. Based on this, after the image mask is convolved by the convolution processing module, a 64-channel feature map with a resolution of 800*800 can also be obtained. Then, the two 64-channel feature maps can be added bit by bit, and other subsequent calculations in the model can be performed.

[0115] In other subsequent calculation processes, pixels corresponding to the area with a grayscale value of 128 on the image mask can be identified, and the loss function can be calculated based on these pixels until the second image semantic segmentation model trained can better segment the shoes and shadow areas from the target area with high accuracy.

[0116] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 201 to 204 can be device A; for another example, the execution subject of steps 201 and 202 can be device A, and the execution subject of step 203 can be device B; and so on.

[0117] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 201, 202, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0118] Figure 4 is a schematic diagram of the structure of an image processing device provided by an exemplary embodiment of the present application. Figure 4 As shown, the image processing device includes: a memory 401 and a processor 402 .

[0119] The memory 401 is used to store computer programs and can be configured to store various other data to support operations on the image processing device. Examples of such data include instructions for any application or method operating on the image processing device, contact data, phone book data, messages, pictures, videos, etc.

[0120] Among them, the memory 401 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0121] The processor 402 is coupled to the memory 401 and is used to execute the computer program in the memory 401, so as to: segment the image to be processed to obtain a first estimated area corresponding to the first target image and a second estimated area corresponding to the second target image; determine the target area on the image to be processed that includes the boundary of the first target image and the second target image according to the contour features of the first estimated area and the second estimated area; and re-segment the target area on the image to be processed to correct the first estimated area and the second estimated area.

[0122] Further optionally, when the processor 402 segments the image to be processed and obtains the first estimated area corresponding to the first target image and the second estimated area corresponding to the second target image, it is specifically used to: input the image to be processed into a first image segmentation model; the first image segmentation model uses sample images including the first target image and the second target image as training samples, and is trained by minimizing the loss function of the first target image and / or the second target image; and runs the first image segmentation model to identify the first estimated area and the second estimated area from the image to be processed.

[0123] Further optionally, when the processor 402 determines the target area on the image to be processed that includes the boundary of the first target image and the second target image based on the contour features of the first estimated area and the second estimated area, the processor 402 is specifically used to: expand the contour of the first estimated area to obtain a first extended contour, and expand the contour of the second estimated area to obtain a second extended contour; obtain the overlapping area of ​​the first extended contour and the second extended contour; and use the image area on the image to be processed that corresponds to the overlapping area as the target area on the image to be processed.

[0124] Further optionally, when the processor 402 re-segments the target area on the image to be processed to correct the first estimated area and the second estimated area, it is specifically used to: identify some pixels belonging to the first target image and some pixels belonging to the second target image in the target area on the image to be processed to re-segment the target area on the image to be processed; and correct the first estimated area and the second estimated area according to the result of the re-segmentation to obtain the first target image and the second target image.

[0125] Further optionally, when the processor 402 identifies some pixels belonging to the first target image and some pixels belonging to the second target image in the target area on the image to be processed to re-segment the target area on the image to be processed, it is specifically used to: mark the target area on the image to be processed; input the marked image to be processed into a second image segmentation model; the second image segmentation model is trained by minimizing the loss function of the target area using sample images marked with the first target image and the second target image and marked with the target area as training samples; and the second image segmentation model is run to identify some pixels belonging to the first target image and some pixels belonging to the second target image from the target area on the image to be processed.

[0126] Further optionally, when marking the target area on the image to be processed, the processor 402 is specifically used to: generate a first image mask with the same resolution as the image to be processed; in the first image mask, the grayscale value of the pixel corresponding to the target area on the image to be processed is a set marking value, so as to mark the target area on the image to be processed by the marking value.

[0127] Further optionally, when the processor 402 runs the second image segmentation model to identify partial pixels belonging to the first target image and partial pixels belonging to the second target image from the target area on the image to be processed, it is specifically used to: within the second image segmentation model, use a single-channel first convolution kernel provided by a mask convolution module to perform convolution processing on the first image mask to obtain a first feature map; and, in the first convolution layer, use a three-channel second convolution kernel to perform convolution processing on the image to be processed to obtain a second feature map; the first convolution kernel and the second convolution kernel have the same size and number; bitwise superimpose the first feature map and the second feature map to obtain a third feature map; and input the third feature map into other computing layers after the first convolution layer, so that the other computing layers identify partial pixels belonging to the first target image and partial pixels belonging to the second target image in the target area on the image to be processed according to the third feature map.

[0128] Further optionally, when the processor 402 corrects the first estimated area and the second estimated area according to the re-segmentation result to obtain the first target image and the second target image, it is specifically used to: replace the image area corresponding to the target area on the image to be processed in the first estimated area and the second estimated area with the partial pixels belonging to the first target image and the partial pixels belonging to the second target image to obtain the first target image and the second target image.

[0129] Further optionally, the processor 402 is also used to: obtain a sample image labeled with a first target image and a second target image, and generate a second image mask with the same resolution as the sample image according to the target area on the sample image; input the sample image and the second image mask into the neural network model to perform the following model training operations: use a single-channel first convolution kernel provided by the mask convolution module to perform convolution processing on the second image mask to obtain a first feature sample map; and use a three-channel second convolution kernel to perform convolution processing on the sample image in the first convolution layer to obtain a second feature sample map; the first convolution kernel and the second convolution kernel are the same in size and number; bitwise The first feature sample map and the second feature sample map are superimposed to obtain a third feature sample map, and the third feature sample map is input into other computing layers after the first convolution layer; the recognition result of the target area on the sample image by the other computing layer is obtained according to the third feature sample map, and the loss function of the target area is calculated; it is determined whether the loss function of the target area is less than a set threshold value, and if so, the model parameters of the second image segmentation model are determined according to the parameters of the first convolution layer and the other computing layers; if not, the parameters of the mask convolution module, the first convolution layer and the other computing layers are updated, and the model training operation is re-executed.

[0130] Further, if Figure 4 As shown, the image processing device also includes: a communication component 403, a display 404, a power component 405, an audio component 406 and other components. Figure 4 Only some components are shown schematically, which does not mean that the image processing device only includes Figure 4 Components shown.

[0131] Among them, the communication component 403 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0132] Among them, the display 404 includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0133] The power supply component 405 provides power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply component is located.

[0134] In this embodiment, the image to be processed is segmented to obtain a first estimated area corresponding to the first target image and a second estimated area corresponding to the second target image. According to the contour features of the two estimated image areas, the target area on the image to be processed that includes the boundary of the first target image and the second target image can be determined; then, the target area on the image to be processed is segmented to correct the first estimated area and the second estimated area. Based on the above image processing process, after obtaining the segmentation result of the image to be processed, the segmentation result of the image to be processed is corrected according to the segmentation result of the target area on the image to be processed that is not easy to accurately segment, thereby improving the segmentation accuracy of the image and effectively increasing the availability and reliability of the image segmentation result.

[0135] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be performed by the image processing device in the above method embodiment.

[0136] Figure 5 is a schematic diagram of the structure of an image processing device provided by an exemplary embodiment of the present application. Figure 5 As shown, the image processing device includes: a memory 501 and a processor 502 .

[0137] The memory 501 is used to store computer programs and can be configured to store various other data to support operations on the image processing device. Examples of such data include instructions for any application or method operating on the image processing device, contact data, phone book data, messages, pictures, videos, etc.

[0138] Among them, the memory 501 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0139] The processor 502 is coupled to the memory 501 and is used to execute the computer program in the memory 501, so as to: segment the image to be processed to obtain an estimated subject area corresponding to the subject on the image to be processed and an estimated shadow area corresponding to the shadow of the subject; determine a target area on the image to be processed that includes a boundary between the subject and the shadow of the subject according to contour features of the estimated subject area and the estimated shadow area; and re-segment the target area to obtain the subject and the shadow of the subject on the image to be processed.

[0140] Further optionally, when the processor 502 re-segments the target area to obtain the main object and the shadow of the main object on the image to be processed, it is specifically used to: identify some pixels in the target area that belong to the estimated main area and some pixels that belong to the estimated shadow area to re-segment the main object on the image to be processed; and correct the estimated main area and the estimated shadow area according to the result of the re-segmentation to obtain the main object and the shadow of the main object on the image to be processed.

[0141] Further, if Figure 5 As shown, the image processing device also includes: a communication component 503, a display 504, a power component 505, an audio component 506 and other components. Figure 5 Only some components are shown schematically, which does not mean that the image processing device only includes Figure 5 Components shown.

[0142] Among them, the communication component 503 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0143] Among them, the display 504 includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0144] The power supply component 505 provides power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply component is located.

[0145] Based on the technical solution of this embodiment, the main object and its shadow on the image to be processed can be segmented for the first time to obtain the estimated main area and the estimated shadow area. Then, according to the contour features of the estimated main area and the estimated shadow area, the target area including the boundary between the main object and its shadow is determined from the image to be processed; then, the target area on the image to be processed is segmented again to correct the result of the first segmentation, so that the main object and its shadow obtained by the final segmentation have higher accuracy, effectively increasing the availability and reliability of the image segmentation result.

[0146] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be performed by the image processing device in the above method embodiment.

[0147] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0148] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0149] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0150] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0151] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0152] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0153] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0154] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0155] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An image processing method, characterized in that: include: Segmenting the image to be processed to obtain an estimated main body region corresponding to the main object on the image to be processed and an estimated shadow region corresponding to the shadow of the main object; Extending the estimated outline of the main body area to obtain a first extended outline, and extending the estimated outline of the shadow area to obtain a second extended outline; Acquire an overlapping area of ​​the first extended outline and the second extended outline, and use an image area on the image to be processed corresponding to the overlapping area as a target area on the image to be processed; Identify some pixels in the target area belonging to the estimated main area and some pixels belonging to the estimated shadow area, so as to re-segment the target area on the image to be processed; The estimated main body region and the estimated shadow region are corrected according to the result of the re-segmentation to obtain the main object and the shadow of the main object on the image to be processed.

2. An image processing method, characterized in that: include: Segmenting the image to be processed to obtain a first estimated region corresponding to the first target image and a second estimated region corresponding to the second target image; Extending the outline of the first estimated area to obtain a first extended outline, and extending the outline of the second estimated area to obtain a second extended outline; Acquire an overlapping area of ​​the first extended outline and the second extended outline, and use an image area on the image to be processed corresponding to the overlapping area as a target area on the image to be processed; Identify some pixels in the target area belonging to the first estimated area and some pixels belonging to the second estimated area, so as to re-segment the target area on the image to be processed; The first estimated region and the second estimated region are modified according to the result of the re-segmentation.

3. The method according to claim 2, characterized in that Segmenting the image to be processed to obtain a first estimated region corresponding to the first target image and a second estimated region corresponding to the second target image, including: Inputting the image to be processed into a first image segmentation model; the first image segmentation model uses sample images including a first target image and a second target image as training samples, and is trained by minimizing a loss function of the first target image and / or the second target image; The first image segmentation model is run to identify the first estimated region and the second estimated region from the image to be processed.

4. The method according to claim 2, characterized in that: Identifying a portion of pixels belonging to the first target image and a portion of pixels belonging to the second target image in the target area on the image to be processed, so as to re-segment the target area on the image to be processed, comprising: Marking a target area on the image to be processed; Inputting the marked image to be processed into a second image segmentation model; the second image segmentation model uses sample images marked with the first target image and the second target image and marked with the target area as training samples, and is trained by minimizing the loss function of the target area; The second image segmentation model is run to identify some pixels belonging to the first target image and some pixels belonging to the second target image from the target area on the image to be processed.

5. The method according to claim 4, characterized in that Marking the target area on the image to be processed includes: A first image mask having the same resolution as the image to be processed is generated; in the first image mask, the grayscale value of the pixel corresponding to the target area on the image to be processed is a set mark value, so as to mark the target area on the image to be processed by the mark value.

6. The method according to claim 5, characterized in that Running the second image segmentation model to identify some pixels belonging to the first target image and some pixels belonging to the second target image from the target area on the image to be processed, comprising: Within the second image segmentation model, a single-channel first convolution kernel provided by a mask convolution module is used to perform convolution processing on the first image mask to obtain a first feature map; and, in the first convolution layer, a three-channel second convolution kernel is used to perform convolution processing on the image to be processed to obtain a second feature map; the first convolution kernel and the second convolution kernel are the same in size and number; Bitwise superposition of the first feature map and the second feature map to obtain a third feature map; The third feature map is input into other computing layers after the first convolutional layer, so that the other computing layers identify some pixels belonging to the first target image and some pixels belonging to the second target image in the target area on the image to be processed according to the third feature map.

7. The method according to any one of claims 2 to 6, characterized in that: Modifying the first estimated region and the second estimated region according to the re-segmentation result to obtain the first target image and the second target image includes: The first target image and the second target image are obtained by replacing the image areas on the first estimated area and the second estimated area corresponding to the target area on the image to be processed with the partial pixels belonging to the first target image and the partial pixels belonging to the second target image.

8. The method according to any one of claims 4 to 6, characterized in that: Also includes: Acquire a sample image annotated with a first target image and a second target image, and generate a second image mask having the same resolution as the sample image according to a target area on the sample image; Inputting the sample image and the second image mask into the neural network model to perform the following model training operations: using a single-channel first convolution kernel provided by a mask convolution module to perform convolution processing on the second image mask to obtain a first feature sample image; and, in the first convolution layer, using a three-channel second convolution kernel to perform convolution processing on the sample image to obtain a second feature sample image; the first convolution kernel and the second convolution kernel have the same size and number; Superimposing the first feature sample graph and the second feature sample graph bit by bit to obtain a third feature sample graph, and inputting the third feature sample graph into other computing layers after the first convolutional layer; Obtaining recognition results of the target area on the sample image by the other computing layer according to the third feature sample image, and calculating a loss function of the target area; Determine whether the loss function of the target area is less than a set threshold. If so, determine the model parameters of the second image segmentation model according to the parameters of the first convolution layer and the other computing layers; if not, update the parameters of the mask convolution module, the first convolution layer and the other computing layers, and re-execute the model training operation.

9. An image processing device, characterized in that: include: Memory and processor; Wherein, the memory is used to store one or more computer instructions; The processor is used to execute the one or more computer instructions to: execute the steps in the image processing method according to any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed, the image processing method according to any one of claims 1 to 8 can be implemented.

Images