Image processing method and device, electronic equipment and storage medium

Abstract

The invention discloses an image processing method and device, electronic equipment and a storage medium, and relates to the technical field of computers, in particular to the field of artificial intelligence such as computer vision and deep learning. According to the specific implementation scheme, a target image is segmented to obtain a first mask image of a first area and a second mask image of a second area; determining a boundary area corresponding to an original boundary between the first mask image and the second mask image; performing expansion processing according to the boundary region to obtain a third mask image transited from the first region to the second region and a fourth mask image transited from the second region to the first region; and processing the first mask image and the second mask image according to the third mask image and the fourth mask image.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to artificial intelligence fields such as computer vision and deep learning, and specifically to an image processing method, device, electronic device and storage medium. Background Art

[0002] In the field of image processing, it is a common image processing method to distinguish different regions in the original image and process different regions or combinations of different regions based on mask images of different regions. Summary of the invention

[0003] The present application provides an image processing method, device, electronic device and storage medium. The specific scheme is as follows:

[0004] According to one aspect of the present application, there is provided an image processing method, comprising:

[0005] Segmenting the target image to obtain a first mask image of the first area and a second mask image of the second area;

[0006] determining a boundary area corresponding to an original boundary between the first mask image and the second mask image;

[0007] Performing dilation processing according to the boundary area to obtain a third mask image transitioning from the first area to the second area and a fourth mask image transitioning from the second area to the first area;

[0008] The first mask image and the second mask image are processed according to the third mask image and the fourth mask image.

[0009] According to another aspect of the present application, there is provided an image processing device, comprising:

[0010] A segmentation processing module, used for performing segmentation processing on the target image to obtain a first mask image of the first area and a second mask image of the second area;

[0011] a determination module, configured to determine a boundary area corresponding to an original boundary between the first mask image and the second mask image;

[0012] A first processing module, configured to perform dilation processing according to the boundary area to obtain a third mask image transitioning from the first area to the second area and a fourth mask image transitioning from the second area to the first area;

[0013] The second processing module is configured to process the first mask image and the second mask image according to the third mask image and the fourth mask image.

[0014] According to another aspect of the present application, there is provided an electronic device, including:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the method described in the above embodiment.

[0018] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method according to the above embodiment.

[0019] According to another aspect of the present application, a computer program product is provided, including a computer program, wherein the computer program implements the steps of the method described in the above embodiment when executed by a processor.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0022] Figure 1 A flowchart of an image processing method provided in one embodiment of the present application;

[0023] Figure 2 A flowchart of an image processing method provided by another embodiment of the present application;

[0024] Figure 3 A schematic diagram of a mask image transitioning from one region to another region provided for this application;

[0025] Figure 4 A flowchart of an image processing method provided by another embodiment of the present application;

[0026] Figure 5 A flowchart of an image processing method provided by another embodiment of the present application;

[0027] Figure 6 A schematic diagram of the structure of an image processing device provided in one embodiment of the present application;

[0028] Figure 7It is a block diagram of an electronic device used to implement the image processing method of an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0030] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.

[0031] The following describes the image processing method, device, electronic device and storage medium of the embodiments of the present application with reference to the accompanying drawings.

[0032] Figure 1 A flowchart of an image processing method provided in one embodiment of the present application.

[0033] The image processing method of the embodiment of the present application may be executed by the image processing device of the embodiment of the present application, and the device may be configured in an electronic device.

[0034] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0035] like Figure 1 As shown, the image processing method includes:

[0036] Step 101 : segmenting a target image to obtain a first mask image of a first area and a second mask image of a second area.

[0037] In the present application, the target image may be any image to be processed. For example, the target image may be a single image shot, or may be a frame of an image in a video, which is not limited.

[0038] Exemplarily, a segmentation model may be used to perform segmentation processing on the target image to obtain a first mask image of a first area in the target image and a second mask image of a second area in the target image.

[0039] The first area and the second area may be different areas in the target image, and the first area is adjacent to the second area.

[0040] The first mask image and the second mask image may be binary images, grayscale images, etc. Taking a binary image as an example, the pixel values ​​in the first area of ​​the first mask image may be assigned a value of 255, and the pixel values ​​in other areas except the first area may be assigned a value of 0. The pixel values ​​in the second area of ​​the second mask image may be assigned a value of 255, and the pixel values ​​in other areas except the second area may be assigned a value of 0. Alternatively, the first area of ​​the first mask image may be marked as 1, and other areas except the first area may be marked as 0.

[0041] Exemplarily, the first mask image and the second mask image may be obtained by performing segmentation processing using different or the same segmentation models.

[0042] For example, a skin segmentation model can be used to detect an image containing a face to obtain a mask image of the body skin and a mask image of the face skin. A face detection model can be used to detect an image containing a face to obtain the position information of the face, and a face image can be obtained based on the position information of the face. A facial feature segmentation model can be used to segment the face image to obtain a mask image of the facial features. Thus, different segmentation models can be used to detect different parts.

[0043] Step 102: Determine a boundary area corresponding to an original boundary between the first mask image and the second mask image.

[0044] In the present application, the boundary area may be determined according to the original boundary between the first mask image and the second mask image.

[0045] Illustratively, the boundary area may include the original boundary, partial areas on both sides of the original boundary, and the like.

[0046] Step 103 , performing dilation processing according to the boundary area to obtain a third mask image transitioning from the first area to the second area and a fourth mask image transitioning from the second area to the first area.

[0047] Since the edge of the first mask image or the second mask image may be dislocated with the adjacent position, or the first mask image and the second mask image are superimposed, the boundary position after superposition may also be dislocated, which affects the subsequent processing. Based on this, in the present application, the third mask image can be obtained by dilating from the first area to the second area according to the boundary area, and the fourth mask image can be obtained by dilating from the second area to the first area according to the boundary area.

[0048] The area range of the third mask image and the area range of the fourth mask image are the same as the area range of the boundary area.

[0049] Exemplarily, the mask value in the third mask image may gradually decrease in a direction from the first region to the second region, and the mask value in the fourth mask image may gradually decrease in a direction from the second region to the first region.

[0050] Step 104: Process the first mask image and the second mask image according to the third mask image and the fourth mask image.

[0051] In the present application, the first mask image and the second mask image may be processed according to the third mask image and the fourth mask image based on the processing strategy for the first mask image and the second mask image.

[0052] The processing of the first mask image and the second mask image may include, for example, defect repair, skin smoothing, skin color adjustment, etc., which is not limited and can be processed according to actual needs.

[0053] Among them, the processing strategies may include separate processing, simultaneous processing, etc.

[0054] For example, if the processing strategy is separate processing, the first mask image and the second mask image are processed separately according to the mask images corresponding to the first mask image and the second mask image respectively obtained based on the dilation processing.

[0055] In an embodiment of the present application, by determining the boundary area corresponding to the original boundary between the first mask image of the first area and the second mask image of the second area in the target image, dilation processing is performed according to the boundary area to obtain a mask image for transition from the first area to the second area and a mask image for transition from the second area to the first area, and based on the mask images with two different transition directions, the first mask image and the second mask image are processed, so that the boundary transition between the first mask image and the second mask image can be natural, thereby improving the accuracy of image retouching.

[0056] Figure 2 A flowchart of an image processing method provided in another embodiment of the present application.

[0057] like Figure 2 As shown, the image processing method includes:

[0058] Step 201 : segmenting a target image to obtain a first mask image of a first area and a second mask image of a second area.

[0059] In the present application, step 201 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0060] Step 202: determine a boundary area corresponding to an original boundary between the first mask image and the second mask image.

[0061] For example, the boundary region may be determined by the region of the first mask image and the second mask image that is away from the original boundary by a preset width, thereby accurately controlling which regions will be expanded to enhance the edge effect.

[0062] In the present application, step 201-step 202 can be implemented in any of the embodiments of the present application, so they will not be described in detail here.

[0063] Step 203: remove the boundary area from the first mask image and the second mask image to obtain a fifth mask image corresponding to the first mask image.

[0064] The fifth mask image is the mask image remaining after the boundary area is removed from the first mask image. It can be seen that the fifth mask image is a partial mask image in the first mask image.

[0065] In some embodiments, the boundary area may include two areas, namely, a first sub-area located in the first mask image and a second sub-area located in the second mask image. The boundary area is removed from the first mask image and the second mask image, that is, the first sub-area is removed from the first mask image to obtain the fifth mask image, and the second sub-area is removed from the second mask image to obtain the sixth mask image.

[0066] Step 204 , starting from the first boundary between the fifth mask image and the boundary area toward the second mask image, the fifth mask image is expanded to obtain a third mask image.

[0067] As a possible implementation, starting from the first boundary between the fifth mask image and the boundary area toward the second mask image, pixel points in the boundary area are assigned values ​​in sequence to obtain a third mask image, wherein the mask values ​​of the pixel points in the third mask image decrease in sequence from the first boundary toward the second mask image.

[0068] As another possible implementation, the fifth mask image is expanded from the first boundary to the second mask image to obtain multiple narrow strips until the second boundary between the sixth mask image and the boundary area, and the multiple narrow strips are assigned values ​​to obtain the third mask image. That is, from the first boundary to the second mask image, narrow strips are obtained in sequence until the second boundary.

[0069] The mask values ​​of the plurality of narrow strip areas decrease in sequence from the first boundary to the second mask image.

[0070] The mask values ​​of the pixels in the same narrow strip area may be the same, or may decrease from the first boundary to the second mask image, which is not limited.

[0071] Taking the same mask value of pixels in the same narrow strip area as an example, the mask values ​​of multiple narrow strip areas may be decreased in sequence by reducing a preset value each time, or may be decreased in sequence by a different value each time, and there is no limitation on this.

[0072] For example, the mask value of the pixel in the first area in the fifth mask image is 1, and the mask values ​​corresponding to the five narrow strip areas obtained in sequence are 1, 0.8, 0.6, 0.4, and 0.2, or the corresponding mask values ​​are 1, 0.9, 0.7, 0.6, and 0.4, respectively.

[0073] For example, the fifth mask image can be expanded from the first boundary toward the second mask image according to the set step length until the second original boundary between the boundary area and the sixth mask image is reached, thereby obtaining a plurality of narrow strip areas with the same pixel width. Since the narrow strip areas are obtained according to the set step length, the pixel width of the narrow strip areas is the same as the set step length, and thus the pixel widths of different narrow strip areas are also the same, and the transition is more natural.

[0074] Exemplarily, the fifth mask image may be expanded from the first boundary toward the second mask image according to different step lengths to obtain a plurality of narrow strip areas.

[0075] Thus, expansion is performed starting from the first boundary toward the second mask image to obtain a plurality of narrow strip areas, and the narrow strip areas are assigned values ​​in a descending manner according to the mask values, so that at the boundary between the first mask image and the second mask image, a transition area is constructed where the mask value gradually changes from the first mask image to the second mask image, so that the boundary transition is natural.

[0076] Exemplarily, the fifth mask area can be expanded according to a set step size, starting from the first boundary toward the second mask image until the second original boundary between the boundary area and the sixth mask image is reached, to obtain multiple narrow strip areas, and the number of steps is determined according to the pixel width of the boundary area and the set step size, and the transition band coefficient array is determined according to the number of steps, and the element value corresponding to the narrow strip area in the transition band coefficient array is determined according to the step order corresponding to the narrow strip area, and the element value corresponding to the narrow strip area is used as the mask value of the pixel point in the narrow strip area to obtain a third mask image.

[0077] The number of steps may be determined based on the ratio of the pixel width to the set step length. The pixel width of the narrow strip area is equal to the set step length. Each stepping results in a narrow strip area. Then the number of narrow strip areas is the same as the number of steps.

[0078] The number of elements in the transition zone coefficient array is the same as the number of steps, and the element values ​​in the transition zone coefficient array decrease in sequence.

[0079] The stepping sequence may refer to the number of steps in which the narrow strip area is acquired.

[0080] For example, the number of steps is equal to 5, and the transition band coefficient array is [1, 0.8, 0.6, 0.4, 0.2]. Then, from the first boundary to the second mask image, the stepping order of the narrow strip area obtained by the first step is 1. Then the first value in the transition band coefficient array can be assigned to the first narrow strip area, that is, the mask value of the pixel point in the narrow strip area obtained by the first step is assigned to 1. By analogy, the mask value of the pixel point in the narrow strip area obtained by the second step is assigned to 0.8, the mask value of the pixel point in the narrow strip area obtained by the third step is assigned to 0.6, the mask value of the pixel point in the narrow strip area obtained by the fourth step is assigned to 0.4, and the mask value of the pixel point in the narrow strip area obtained by the fifth step is assigned to 0.2.

[0081] for example, Figure 3 In the middle step-shaped area, the first mask image is expanded in the direction of the second mask image according to the set step length to obtain a narrow strip area, and the third mask image is obtained by assigning the transition band coefficient array.

[0082] Therefore, for multiple narrow strip areas obtained according to the set step size, the transition band coefficient array can be determined according to the pixel width of the boundary area and the set step size, and the values ​​of the corresponding positions in the transition band coefficient array are assigned to the narrow strip areas according to the stepping order corresponding to the narrow strip areas, so as to construct a transition area in the boundary area where the mask value decreases successively from the first mask image to the second mask image, so that the transition is natural.

[0083] Optionally, the mask image obtained by using the element value corresponding to the narrow strip area as the mask value of the pixel point in the narrow strip area can be used as the first initial mask image, and then the first initial mask image can be smoothed to obtain the third mask image. Exemplarily, the third mask image is obtained by performing Gaussian filtering on the first initial mask image to achieve smoothing. Thereby, the transition effect of the third mask image can be improved.

[0084] Step 205 , performing guided filtering on the mask image obtained by superimposing the first mask image and the second mask image to obtain a filtered mask image.

[0085] In the present application, the first mask image and the second mask image may be superimposed to obtain a superimposed mask image, and the superimposed mask image may be subjected to guided filtering to obtain a filtered mask image. Thus, the whole is smoothed by guided filtering, so that there is no obvious boundary between the first mask image and the second mask image.

[0086] Exemplarily, the confidence of the first mask image and the second mask image may be output according to the segmentation model, and the first mask image and the second mask image may be superimposed. The segmentation models of the first mask image and the second mask image may be the same or different, which is not limited.

[0087] For example, a mask image of facial skin is obtained using a skin segmentation model, and a mask image of eyebrows is obtained using a facial features segmentation model. The mask image of facial skin and the mask image of eyebrows can be superimposed based on the confidence of the mask image of facial skin output by the skin segmentation model and the confidence of the mask image of eyebrows output by the facial features segmentation model.

[0088] Therefore, superimposing the confidence results of the segmentation model can improve the accuracy of the superimposed mask image.

[0089] Since the mask image after superposition may still have inaccurate segmentation of mask images in different areas, such as inaccurate facial skin boundaries or facial features boundaries, there are black edges or white edges. The black edge phenomenon refers to the existence of pixel edges with a value close to 255 at the boundary, and the white edge phenomenon refers to the existence of pixel edges with a value close to 0 at the boundary.

[0090] Based on this, if there are pixels with grayscale values ​​within a preset range at the original boundary in the superimposed mask image, the stretching center point can be determined according to the average grayscale value of the pixels in the superimposed mask image, and the difference between the grayscale value of each pixel in the superimposed mask image and the stretching center point can be determined. According to the product of the difference and the stretching weight, the intermediate mask image can be determined, and then the intermediate mask image can be guided filtered to obtain the filtered mask image.

[0091] For example, the preset range may be (253, 255] or [0, 1]. It should be noted that the preset range may be determined according to actual needs and is not limited thereto.

[0092] Exemplarily, the following formula may be used to process the superimposed mask images to obtain an intermediate mask image:

[0093]

[0094] Among them, image is the grayscale value of the pixel in the superimposed mask image; k is the stretching weight; x0 is the stretching center point, which is obtained by counting the average grayscale value of the pixels in image; stretched is the grayscale value of the pixel in the intermediate mask image obtained after processing.

[0095] Therefore, when white edges or black edges appear at the boundaries of the superimposed mask image, the grayscale value of each pixel in the superimposed mask image and the stretching center point can be used to process the superimposed mask image, so as to make the boundary transition sharp and fade the black edges or white edges, thereby improving the quality of the filtered mask image.

[0096] Optionally, a sigmoid function may be used to process the superimposed mask image, thereby fading the black edge or the white edge.

[0097] Step 206 , obtaining a fourth mask image according to the difference between the mask image corresponding to the boundary area in the filtered mask image and the third mask image.

[0098] In the present application, the third mask image may be subtracted from the mask image corresponding to the boundary area in the filtered mask image to obtain a fourth mask image.

[0099] Step 207: Process the first mask image and the second mask image according to the third mask image and the fourth mask image.

[0100] As a possible implementation, if the first mask image and the second mask image are processed simultaneously, the third mask image and the fifth mask image can be superimposed to obtain the seventh mask image, the fourth mask image and the sixth mask image can be superimposed to obtain the eighth mask image, and the seventh mask image and the eighth mask image can be superimposed to obtain the target mask image, and the target mask image is processed. Thus, the two mask images with opposite transition directions obtained based on the expansion process are superimposed, so that when the first mask image and the second mask image are processed simultaneously, there is no obvious fault at the boundary and the transition is natural.

[0101] As a possible implementation method, if the first mask image and the second mask image are processed separately, the third mask image and the fifth mask image can be superimposed to obtain the seventh mask image, and the seventh mask image can be processed, and the fourth mask image and the sixth mask image can be superimposed to obtain the eighth mask image, and the eighth mask image can be processed.

[0102] Therefore, the first mask image is processed separately based on the mask image for transition from the first area to the second area, and the second mask image is processed separately based on the mask image for transition from the second area to the first area, so that when the first mask image and the second mask image are processed separately, there is no obvious discontinuity at the boundary and the transition is natural.

[0103] In an embodiment of the present application, a fifth mask image corresponding to the first mask image is obtained by removing the boundary area, and the fifth mask image is expanded from the first boundary between the fifth mask image and the boundary area toward the second mask image to obtain a third mask image, so that the transition from the first mask image to the second mask image is natural, and a fourth mask image is obtained based on the mask image after the overall guided filtering and the third mask image, which can reduce the amount of calculation and improve the calculation efficiency.

[0104] Figure 4 A flowchart of an image processing method provided in another embodiment of the present application.

[0105] like Figure 4 As shown, the image processing method includes:

[0106] Step 401 : segmenting a target image to obtain a first mask image of a first area and a second mask image of a second area.

[0107] Step 402: Determine a boundary area corresponding to an original boundary between the first mask image and the second mask image.

[0108] In the present application, step 401-step 402 can be implemented in any of the embodiments of the present application, so they will not be described in detail here.

[0109] Step 403: remove the boundary area from the first mask image and the second mask image to obtain a sixth mask image corresponding to the second mask image.

[0110] The sixth mask image is the mask image remaining after the boundary area is removed from the second mask image. It can be seen that the sixth mask image is a partial mask image in the second mask image.

[0111] In some embodiments, the boundary area may include two areas, namely, a first sub-area located in the first mask image and a second sub-area located in the second mask image. The boundary area is removed from the first mask image and the second mask image, that is, the first sub-area is removed from the first mask image to obtain the fifth mask image, and the second sub-area is removed from the second mask image to obtain the sixth mask image.

[0112] Step 404 , starting from the second boundary between the sixth mask image and the boundary area toward the first mask image, the sixth mask image is expanded to obtain a fourth mask image.

[0113] As a possible implementation, starting from the second boundary between the sixth mask image and the boundary area toward the first mask image, pixel points in the boundary area are assigned values ​​in sequence to obtain a fourth mask image, wherein the mask values ​​of the pixel points in the fourth mask image decrease in sequence from the second boundary toward the first mask image.

[0114] As another possible implementation, the sixth mask image is expanded from the second boundary toward the first mask image until the first boundary between the boundary area and the fifth mask image is reached, a plurality of narrow strip areas are obtained, and values ​​are assigned to the plurality of narrow strip areas respectively to obtain a fourth mask image. That is, the narrow strip areas are sequentially obtained from the second boundary toward the first mask image until the second boundary is reached.

[0115] The mask values ​​of the plurality of narrow strip areas decrease in sequence from the second boundary toward the first mask image.

[0116] The detailed process of respectively assigning values ​​to the plurality of narrow strips to obtain the fourth mask image can be found in the description of respectively assigning values ​​to the plurality of narrow strips to obtain the third mask image in the above implementation, which will not be repeated here.

[0117] Thus, expansion is performed starting from the second boundary in the direction of the first mask image to obtain a plurality of narrow strip areas, and the narrow strip areas are assigned values ​​in a descending manner according to the mask values, so that at the boundary between the first mask image and the second mask image, a transition area is constructed in which the mask value gradually changes from the second mask image to the first mask image, so that the boundary transition is natural.

[0118] Step 405 , performing guided filtering on the mask image obtained by superimposing the first mask image and the second mask image to obtain a filtered mask image.

[0119] In the present application, step 405 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0120] Step 406 : Obtain a third mask image according to the difference between the mask image corresponding to the boundary area in the filtered mask image and the fourth mask image.

[0121] In the present application, the third mask image may be obtained by subtracting the fourth mask image from the mask image corresponding to the boundary area in the filtered mask image.

[0122] Step 407: Process the first mask image and the second mask image according to the third mask image and the fourth mask image.

[0123] In the present application, step 407 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0124] In an embodiment of the present application, a sixth mask image corresponding to the second mask image is obtained by removing the boundary area, and the sixth mask image is expanded from the second boundary between the sixth mask image and the boundary area toward the first mask image to obtain a third mask image, so that the transition from the second mask image to the first mask image is natural, and the third mask image is obtained based on the mask image after the overall guided filtering and the fourth mask image, which can reduce the amount of calculation and improve the calculation efficiency.

[0125] Figure 5 A flowchart of an image processing method provided in another embodiment of the present application.

[0126] like Figure 5 As shown, the image processing method includes:

[0127] Step 501 : segmenting a target image to obtain a first mask image of a first area and a second mask image of a second area.

[0128] Step 502: Determine a boundary area corresponding to an original boundary between the first mask image and the second mask image.

[0129] Step 503: remove the boundary area from the first mask image and the second mask image to obtain a fifth mask image corresponding to the first mask image and a sixth mask image corresponding to the second mask image.

[0130] In the present application, steps 501 to 503 may be implemented in any of the embodiments of the present application, and therefore will not be described in detail herein.

[0131] Step 504 , starting from the first boundary between the fifth mask image and the boundary area toward the second mask image, the fifth mask image is expanded to obtain a third mask image.

[0132] As an example, the fifth mask image can be expanded according to a set step size, starting from the first boundary toward the second mask image until the second boundary, to obtain multiple narrow strip areas, and the number of steps is determined according to the pixel width of the boundary area and the set step size. According to the number of steps, the transition band coefficient array is determined, and according to the step order corresponding to the narrow strip area, the element value corresponding to the narrow strip area in the transition band coefficient array is determined, and the element value corresponding to the narrow strip area is used as the mask value of the pixel point in the narrow strip area to obtain the third mask image.

[0133] Step 505 , starting from the second boundary between the sixth mask image and the boundary area toward the first mask image, the sixth mask image is expanded to obtain a fourth mask image.

[0134] In step 505, a fourth mask image can be obtained in a similar manner as in step 504. The step length, transition band coefficient array, etc. are set the same as those in step 504.

[0135] Step 506: Process the first mask image and the second mask image according to the third mask image and the fourth mask image.

[0136] In the present application, step 506 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0137] In an embodiment of the present application, the fifth mask image can be expanded from the first boundary between the fifth mask image and the boundary area toward the second mask image to obtain a third mask image, and the sixth mask image can be expanded from the second boundary between the sixth mask image and the boundary area toward the first mask image to obtain a fourth mask image, thereby constructing mask images with different transition directions in the boundary area, thereby improving the accuracy of image processing.

[0138] The image processing method of the embodiment of the present application can be applied to the processing of different parts of the face, and can also be applied to the processing of adjacent parts of different objects.

[0139] In order to implement the above embodiment, the embodiment of the present application also proposes an image processing device. Figure 6 A schematic diagram of the structure of an image processing device provided in one embodiment of the present application.

[0140] like Figure 6 As shown, the image processing device 600 includes:

[0141] The segmentation processing module 610 is used to perform segmentation processing on the target image to obtain a first mask image of the first area and a second mask image of the second area;

[0142] A determination module 620, configured to determine a boundary area corresponding to an original boundary between the first mask image and the second mask image;

[0143] A first processing module 630 is used to perform dilation processing according to the boundary area to obtain a third mask image transitioning from the first area to the second area and a fourth mask image transitioning from the second area to the first area;

[0144] The second processing module 640 is configured to process the first mask image and the second mask image according to the third mask image and the fourth mask image.

[0145] Optionally, the first processing module 630 is used to:

[0146] removing the boundary area from the first mask image and the second mask image to obtain a fifth mask image corresponding to the first mask image;

[0147] Starting from a first boundary between the fifth mask image and the boundary area toward the second mask image, dilating the fifth mask image to obtain the third mask image;

[0148] performing guided filtering on a mask image obtained by superimposing the first mask image and the second mask image to obtain a filtered mask image;

[0149] The fourth mask image is obtained according to a difference between a mask image corresponding to the boundary area in the filtered mask image and the third mask image.

[0150] Optionally, the first processing module 630 is used to:

[0151] Starting from the first boundary toward the second mask image, dilating the fifth mask image to obtain a plurality of narrow strip areas;

[0152] The plurality of narrow strip regions are assigned values ​​respectively to obtain the third mask image; wherein the mask values ​​of the plurality of narrow strip regions decrease in sequence from the first boundary toward the second mask image.

[0153] Optionally, the pixel width of the narrow strip area is equal to the set step size, and the first processing module 630 is used to:

[0154] Determining the number of steps according to the pixel width of the boundary area and the set step length;

[0155] Determine a transition zone coefficient array according to the number of steps; wherein the number of elements in the transition zone coefficient array is the same as the number of steps, and the element values ​​in the transition zone coefficient array decrease in sequence;

[0156] Determining, according to the stepping order corresponding to the narrow strip area, the element value corresponding to the narrow strip area in the transition band coefficient array;

[0157] The element values ​​corresponding to the narrow strip area are used as mask values ​​of the pixels in the narrow strip area to obtain the third mask image.

[0158] Optionally, the first processing module 630 is used to:

[0159] In response to the presence of pixels having grayscale values ​​within a preset range at the original boundary in the superimposed mask image, determining a stretching center point according to an average grayscale value of the pixels in the superimposed mask image;

[0160] Determine the difference between the grayscale value of each pixel in the superimposed mask image and the stretching center point;

[0161] Determine an intermediate mask image according to the product of the difference and the stretching weight;

[0162] Performing guided filtering on the intermediate mask image to obtain the filtered mask image.

[0163] Optionally, the first processing module 630 is used to:

[0164] removing the boundary area from the first mask image and the second mask image to obtain a sixth mask image corresponding to the second mask image;

[0165] Starting from a second boundary between the sixth mask image and the boundary area toward the first mask image, dilating the sixth mask image to obtain the fourth mask image;

[0166] performing guided filtering on a mask image obtained by superimposing the first mask image and the second mask image to obtain a filtered mask image;

[0167] The third mask image is obtained according to a difference between a mask image corresponding to the boundary area in the filtered mask image and the fourth mask image.

[0168] Optionally, the first processing module 630 is used to:

[0169] Starting from the second boundary toward the first mask image, dilating the sixth mask image to obtain a plurality of narrow strip areas;

[0170] The plurality of narrow strip regions are assigned values ​​respectively to obtain the fourth mask image; wherein the mask values ​​of the plurality of narrow strip regions decrease in sequence from the second boundary toward the first mask image.

[0171] Optionally, the second processing module 640 is used to:

[0172] superimposing the third mask image and the fifth mask image to obtain a seventh mask image; wherein the fifth mask image is the mask image remaining from the first mask image after removing the boundary area;

[0173] superimposing the fourth mask image and the sixth mask image to obtain an eighth mask image; wherein the sixth mask image is the mask image remaining from the second mask image after removing the boundary area;

[0174] Superimposing the seventh mask image and the eighth mask image to obtain a target mask image;

[0175] The target mask image is processed.

[0176] Optionally, the second processing module 640 is used to:

[0177] superimposing the third mask image and the fifth mask image to obtain a seventh mask image, and processing the seventh mask image; wherein the fifth mask image is the mask image remaining from the first mask image after removing the boundary area;

[0178] The fourth mask image and the sixth mask image are superimposed to obtain an eighth mask image, and the eighth mask image is processed; wherein the sixth mask image is the mask image remaining from the second mask image after removing the boundary area.

[0179] Optionally, the determining module 620 is configured to:

[0180] An area in the first mask image and the second mask image that is away from the original boundary by a preset width is determined as the boundary area.

[0181] It should be noted that the explanation of the above-mentioned image processing method embodiment is also applicable to the image processing device of this embodiment, so it will not be repeated here.

[0182] In an embodiment of the present application, by determining the boundary area corresponding to the original boundary between the first mask image of the first area and the second mask image of the second area in the target image, dilation processing is performed according to the boundary area to obtain a mask image for transition from the first area to the second area and a mask image for transition from the second area to the first area, and based on the mask images with two different transition directions, the first mask image and the second mask image are processed, so that the boundary transition between the first mask image and the second mask image can be natural, thereby improving the accuracy of image retouching.

[0183] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0184] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0185] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 702 or a computer program loaded from a storage unit 708 to a RAM (Random Access Memory) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An I / O (Input / Output) interface 705 is also connected to the bus 704.

[0186] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0187] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as an image processing method. For example, in some embodiments, the image processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the image processing method in any other appropriate manner (eg, by means of firmware).

[0188] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0189] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0190] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0191] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0192] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0193] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0194] According to an embodiment of the present application, the present application also provides a computer program product, which, when an instruction processor in the computer program product executes, executes the image processing method proposed in the above embodiment of the present application.

[0195] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0196] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. An image processing method, characterized in that: The method comprises: Segmenting the target image to obtain a first mask image of the first area and a second mask image of the second area; determining a boundary area corresponding to an original boundary between the first mask image and the second mask image; Performing dilation processing according to the boundary area to obtain a third mask image transitioning from the first area to the second area and a fourth mask image transitioning from the second area to the first area; The first mask image and the second mask image are processed according to the third mask image and the fourth mask image.

2. The method of claim 1, wherein: The step of performing dilation processing according to the boundary area to obtain a third mask image transitioning from the first area to the second area and a fourth mask image transitioning from the second area to the first area includes: removing the boundary area from the first mask image and the second mask image to obtain a fifth mask image corresponding to the first mask image; Starting from a first boundary between the fifth mask image and the boundary area toward the second mask image, dilating the fifth mask image to obtain the third mask image; performing guided filtering on a mask image obtained by superimposing the first mask image and the second mask image to obtain a filtered mask image; The fourth mask image is obtained according to a difference between a mask image corresponding to the boundary area in the filtered mask image and the third mask image.

3. The method of claim 2, wherein: The step of dilating the fifth mask image from a first boundary between the fifth mask image and the boundary area toward the second mask image to obtain the third mask image includes: Starting from the first boundary toward the second mask image, dilating the fifth mask image to obtain a plurality of narrow strip areas; The plurality of narrow strip regions are assigned values ​​respectively to obtain the third mask image; wherein the mask values ​​of the plurality of narrow strip regions decrease in sequence from the first boundary toward the second mask image.

4. The method of claim 3, wherein: The pixel width of the narrow strip area is equal to the set step size, and the assigning values ​​to the plurality of narrow strip areas respectively to obtain the third mask image includes: Determining the number of steps according to the pixel width of the boundary area and the set step length; Determine a transition zone coefficient array according to the number of steps; wherein the number of elements in the transition zone coefficient array is the same as the number of steps, and the element values ​​in the transition zone coefficient array decrease in sequence; Determining, according to the stepping order corresponding to the narrow strip area, the element value corresponding to the narrow strip area in the transition band coefficient array; The element values ​​corresponding to the narrow strip area are used as mask values ​​of the pixels in the narrow strip area to obtain the third mask image.

5. The method of claim 2, wherein: The step of performing guided filtering on the mask image obtained by superimposing the first mask image and the second mask image to obtain a filtered mask image includes: In response to the presence of pixels having grayscale values ​​within a preset range at the original boundary in the superimposed mask image, determining a stretching center point according to an average grayscale value of the pixels in the superimposed mask image; Determine the difference between the grayscale value of each pixel in the superimposed mask image and the stretching center point; Determine an intermediate mask image according to the product of the difference and the stretching weight; Performing guided filtering on the intermediate mask image to obtain the filtered mask image.

6. The method of claim 1, wherein: The step of performing dilation processing according to the boundary area to obtain a third mask image transitioning from the first area to the second area and a fourth mask image transitioning from the second area to the first area includes: removing the boundary area from the first mask image and the second mask image to obtain a sixth mask image corresponding to the second mask image; Starting from a second boundary between the sixth mask image and the boundary area toward the first mask image, dilating the sixth mask image to obtain the fourth mask image; performing guided filtering on a mask image obtained by superimposing the first mask image and the second mask image to obtain a filtered mask image; The third mask image is obtained according to a difference between a mask image corresponding to the boundary area in the filtered mask image and the fourth mask image.

7. The method of claim 6, wherein: The step of dilating the sixth mask image from a second boundary between the sixth mask image and the boundary area toward the first mask image to obtain the fourth mask image includes: Starting from the second boundary toward the first mask image, dilating the sixth mask image to obtain a plurality of narrow strip areas; The plurality of narrow strip regions are assigned values ​​respectively to obtain the fourth mask image; wherein the mask values ​​of the plurality of narrow strip regions decrease in sequence from the second boundary toward the first mask image.

8. The method according to any one of claims 1 to 7, wherein: The processing of the first mask image and the second mask image according to the third mask image and the fourth mask image includes: superimposing the third mask image and the fifth mask image to obtain a seventh mask image; wherein the fifth mask image is the mask image remaining from the first mask image after removing the boundary area; superimposing the fourth mask image and the sixth mask image to obtain an eighth mask image; wherein the sixth mask image is the mask image remaining from the second mask image after removing the boundary area; Superimposing the seventh mask image and the eighth mask image to obtain a target mask image; The target mask image is processed.

9. The method according to any one of claims 1 to 7, wherein: The processing of the first mask image and the second mask image according to the third mask image and the fourth mask image includes: superimposing the third mask image and the fifth mask image to obtain a seventh mask image, and processing the seventh mask image; wherein the fifth mask image is the mask image remaining from the first mask image after removing the boundary area; The fourth mask image and the sixth mask image are superimposed to obtain an eighth mask image, and the eighth mask image is processed; wherein the sixth mask image is the mask image remaining from the second mask image after removing the boundary area.

10. The method according to any one of claims 1 to 7, wherein: The determining a boundary area corresponding to an original boundary between the first mask image and the second mask image includes: An area in the first mask image and the second mask image that is away from the original boundary by a preset width is determined as the boundary area.

11. An image processing device, characterized in that: The device comprises: A segmentation processing module, used for performing segmentation processing on the target image to obtain a first mask image of the first area and a second mask image of the second area; a determination module, configured to determine a boundary area corresponding to an original boundary between the first mask image and the second mask image; A first processing module, configured to perform dilation processing according to the boundary area to obtain a third mask image transitioning from the first area to the second area and a fourth mask image transitioning from the second area to the first area; The second processing module is configured to process the first mask image and the second mask image according to the third mask image and the fourth mask image.

12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

14. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images