Image processing method, device, electronic device and storage medium

Abstract

The present disclosure relates to an image processing method, device, electronic device and storage medium, and relates to the field of image processing technology, wherein the image processing method comprises: obtaining an image to be processed; extracting semantic features of the image to be processed, and inputting the obtained semantic feature map into a first image segmentation network to determine an object segmentation contour area and a hair segmentation area from the image to be processed; scaling the image to be processed, and inputting the scaled image to be processed and the semantic feature map into a second image segmentation network to determine an edge segmentation area from the image to be processed; and determining a target object segmentation area from the image to be processed according to the object segmentation contour area, the hair segmentation area and the edge segmentation area. Thus, without the need for an additional guide image, a clear hair edge can be obtained, the hair edge sharpness and the hair segmentation effect are improved, the feathering effect is reduced, and refined object segmentation is achieved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] Fine-grained image segmentation technology is a technology that accurately locates and identifies the region of interest in an image. In addition to the original image, most fine-grained image extraction technologies require additional image input guidance. Common additional input images include a background image with the foreground object removed, a rough foreground and background segmentation map, or a three-classification map.

[0003] Since the additional guide image makes the use of refined image segmentation technology more difficult, in the related art, the refined image segmentation technology that does not require additional guide images directly uses the trained image segmentation model to segment the image. However, due to the limitations of the learning ability of the image segmentation model and the precision of sample annotation, the boundaries of the hair area obtained by the image segmentation model are not clear enough, the hair edges are not sharp enough, and there is a strong feathering effect, which cannot achieve the fine hair-level cutout capability. Summary of the invention

[0004] The present disclosure provides an image processing method, device, electronic device and storage medium to at least solve the problem in the related art that the boundary of the hair area obtained by the image segmentation model is not clear enough, the hair edge is not sharp enough, there is a strong feathering effect, and the fine cutout capability of the hair level cannot be achieved. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, comprising: acquiring an image to be processed; performing semantic feature extraction on the image to be processed to obtain a semantic feature map of the image to be processed; inputting the semantic feature map into a first image segmentation network to determine an object segmentation contour area and a hair segmentation area from the image to be processed; scaling the image to be processed, and inputting the scaled image to be processed and the semantic feature map into a second image segmentation network to determine an edge segmentation area from the image to be processed; and determining a target object segmentation area from the image to be processed based on the object segmentation contour area and the hair segmentation area.

[0006] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing device, comprising: a first acquisition module, configured to acquire an image to be processed; a second acquisition module, configured to extract semantic features of the image to be processed to obtain a semantic feature map of the image to be processed; a first determination module, configured to input the semantic feature map into a first image segmentation network to determine an object segmentation contour area and a hair segmentation area from the image to be processed; a second determination module, configured to scale the image to be processed, and input the scaled image to be processed and the semantic feature map into a second image segmentation network to determine an edge segmentation area from the image to be processed; and a third determination module, configured to perform a fusion process on the object segmentation contour area and the edge segmentation area according to the object segmentation contour area and the hair segmentation area to determine a target object segmentation area from the image to be processed.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the image processing method as described in the embodiment of the first aspect of the present disclosure.

[0008] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the image processing method described in the embodiment of the first aspect of the present disclosure.

[0009] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor of an electronic device, the electronic device is enabled to execute the method described in the embodiment of the first aspect of the present disclosure.

[0010] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0011] The semantic feature map of the image to be processed is input into the first image segmentation network to determine the object segmentation contour area and the hair segmentation area from the image to be processed. At the same time, the image to be processed is scaled, and the scaled image to be processed and the semantic feature map are input into the second image segmentation network to determine the edge segmentation area from the image to be processed; then, the target object segmentation area is determined according to the object segmentation contour area, the hair segmentation area and the edge segmentation area. Thus, without the need for additional guide images, clear hair edges can be obtained, the hair edge sharpness and hair segmentation effect are improved, the feathering effect is reduced, and refined object segmentation is achieved.

[0012] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0014] Figure 1 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0015] Figure 2 is a flowchart of another image processing method according to an exemplary embodiment.

[0016] Figure 3 is a schematic diagram showing generating a target object segmentation region according to an exemplary embodiment.

[0017] Figure 4 is a flowchart of another image processing method according to an exemplary embodiment.

[0018] Figure 5 is a flowchart of another image processing method according to an exemplary embodiment.

[0019] Figure 6 is a flowchart of another image processing method according to an exemplary embodiment.

[0020] Figure 7 is a block diagram of an image processing device according to an exemplary embodiment.

[0021] Figure 8 The invention is a block diagram of an electronic device for an image processing method according to an exemplary embodiment. DETAILED DESCRIPTION

[0022] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0024] Fine-grained image segmentation technology is a technology that accurately locates and identifies the region of interest in an image. In addition to the original image, most fine-grained image extraction technologies require additional image input guidance. Common additional input images include a background image with the foreground object removed, a rough foreground and background segmentation map, or a three-classification map.

[0025] Since the additional guide image makes the use of refined image segmentation technology more difficult, the related art mainly includes the following two methods for refined image segmentation technology that does not require additional guide images. The first is to directly use the trained image segmentation model to segment the image; the second is to pre-set the static cutout area and use the original image and the static cutout to solve the cutout weight value. However, due to the limitations of the learning ability of the image segmentation model and the precision of sample annotation, the boundaries of the hair area obtained by the image segmentation model are not clear enough, the hair edges are not sharp enough, and there is a strong feathering effect, which makes it impossible to achieve fine hair-level cutout capabilities; in the second method, the static cutout area needs to be set by the user through an interactive interface, which increases the user's usage cost and limits the usage scenarios.

[0026] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment. Figure 1 As shown, the image processing method can be applied to the image processing device of the embodiment of the present disclosure, and the device can be configured in an electronic device. The electronic device can be a mobile terminal, for example, a mobile phone, a tablet computer, a personal digital assistant, and other hardware devices with various operating systems.

[0027] like Figure 1 As shown, the image processing method comprises the following steps:

[0028] In step 101, an image to be processed is obtained.

[0029] In the embodiment of the present disclosure, an image including a human portrait or an image including an animal head portrait may be used as an image to be processed. As an example, the image to be processed may be obtained through a web crawler, user upload, or network download.

[0030] In step 102, semantic features are extracted from the image to be processed to obtain a semantic feature map of the image to be processed.

[0031] In the disclosed embodiment, the image to be processed may be input into a convolutional neural network, and feature extraction may be performed on the image to be processed to obtain a semantic feature map of the image to be processed.

[0032] In step 103, the semantic feature map is input into the first image segmentation network to determine the object segmentation contour area and the hair segmentation area from the image to be processed.

[0033] Furthermore, the semantic feature map may be input into a first image segmentation network, and the first image segmentation network may output an object segmentation region and a hair segmentation region.

[0034] In step 104, the image to be processed is scaled, and the scaled image to be processed and the semantic feature map are input into a second image segmentation network to determine an edge segmentation region from the image to be processed.

[0035] Optionally, the image to be processed can be scaled to obtain a first scaled image and a second scaled image. For example, the size of the first scaled image is half of the size of the image to be processed, and the size of the second scaled image is one quarter of the size of the image to be processed. Then, a feature map corresponding to the first scaled image is obtained, and the feature map corresponding to the first scaled image, the second scaled image and the semantic feature map are input into a second image segmentation network. The second image segmentation network can output an edge segmentation area. It should be noted that the second image segmentation network can be a refined edge prediction network, which can be a trained convolutional neural network. The refined edge prediction network can output a refined edge segmentation area of ​​the image.

[0036] In step 105, the object segmentation contour region and the edge segmentation region are fused according to the object segmentation contour region and the hair segmentation region to determine the target object segmentation region from the image to be processed.

[0037] In the embodiment of the present disclosure, the intersection area of ​​the object segmentation area and the hair segmentation area can be determined based on the object segmentation area and the hair segmentation area. The intersection area can be the edge area of ​​the hair. The value of each pixel in the intersection area is in the range of (0,1). The value of each pixel in the intersection area can be used as the weight of the edge segmentation area. According to the weight, the object segmentation area and the edge segmentation area are image fused to determine a refined edge segmentation area.

[0038] In summary, the semantic feature map of the image to be processed is input into the first image segmentation network to determine the object segmentation contour area and the hair segmentation area from the image to be processed. At the same time, the image to be processed is scaled, and the scaled image to be processed and the semantic feature map are input into the second image segmentation network to determine the edge segmentation area from the image to be processed; then, the target object segmentation area is determined according to the object segmentation contour area, the hair segmentation area and the edge segmentation area. In this way, a clear hair edge can be obtained without the need for an additional guide image, the hair edge sharpness and the hair segmentation effect are improved, the feathering effect is reduced, and a refined object segmentation is achieved.

[0039] In order to achieve refined object segmentation, such as Figure 2 As shown, Figure 2 This is a flowchart of another image processing method according to an exemplary embodiment. In the disclosed embodiment, the intersection area of ​​the object segmentation area and the hair segmentation area can be determined according to the object segmentation area and the hair segmentation area. The intersection area can be the edge area of ​​the hair. The values ​​of each pixel in the intersection area are in the range of (0, 1). The values ​​of each pixel in the intersection area can be used as the weights of each pixel in the edge segmentation area. According to the weights, the object segmentation area and the edge segmentation area are image fused to determine a refined object segmentation area. Figure 2 The illustrated embodiment may include the following steps:

[0040] Step 201, obtaining an image to be processed.

[0041] Step 202: extract semantic features from the image to be processed to obtain a semantic feature map of the image to be processed.

[0042] Step 203: input the semantic feature map into the first image segmentation network to determine the object segmentation contour area and the hair segmentation area from the image to be processed.

[0043] Step 204, scaling the image to be processed, and inputting the scaled image to be processed and the semantic feature map into a second image segmentation network to determine an edge segmentation region from the image to be processed.

[0044] Step 205: Determine the weight of the edge segmentation area according to the value of each pixel point in the intersection area of ​​the object segmentation contour area and the hair segmentation area.

[0045] Optionally, the object segmentation contour area is corroded to obtain a processed object segmentation contour area; the hair segmentation area is expanded to obtain a expanded hair segmentation area; the processed object segmentation contour area and the processed hair segmentation area are blurred to obtain a blurred object segmentation contour area and a blurred hair segmentation area respectively; the blurred object segmentation contour area is negated to obtain a negated object segmentation contour area; the intersection area of ​​the negated object segmentation contour area and the blurred hair segmentation area is determined, and the value of each pixel point in the intersection area is used as the weight of the edge segmentation area.

[0046] That is to say, the object segmentation area can be corroded, and at the same time, the hair segmentation area can be dilated, and the corroded object segmentation area and the dilated hair segmentation area can be blurred to obtain the blurred object segmentation contour area and the blurred hair segmentation area. For example, the corroded object segmentation area and the dilated hair segmentation area can be Gaussian filtered to obtain the Gaussian filtered object segmentation area and the Gaussian filtered hair segmentation area. Then, the blurred object segmentation contour area can be negated to obtain the negated object segmentation contour area, for example, the foreground of the blurred object segmentation contour area is changed to the background. Then, the intersection area of ​​the negated object segmentation contour area and the blurred hair segmentation area is determined. Since the values ​​of each pixel in the intersection area are in the range of (0, 1), the values ​​of each pixel in the intersection area can be used as the weights of the edge segmentation area.

[0047] Step 206: According to the weight of the edge segmentation region, the object segmentation contour region and the edge segmentation region are merged to generate a target object segmentation region.

[0048] Furthermore, according to the weight of each pixel in the edge segmentation area, the object segmentation contour area and the edge segmentation area are fused, and the fused segmentation area is used as the target object segmentation area. Specifically, it can be expressed as the following formula:

[0049] α*Alpha+(1-α)*Portrait;

[0050] Among them, α is the weight of each pixel in the edge segmentation area, Alpha is the value of each pixel in the edge segmentation area, and Portrait is the value of each pixel in the object segmentation contour area.

[0051] For example, Figure 3As shown, according to the object segmentation contour area Portrait and the hair segmentation area Hair determined from the image to be processed, the weight Alpha Weight of the edge segmentation area Alpha is determined, and according to the weight Alpha Weight of the edge segmentation area, the object segmentation contour area Portrait and the edge segmentation area Alpha are fused, and the fused segmentation area Result is used as the target object segmentation area.

[0052] In the embodiment of the present application, steps 201-204 can be implemented in any of the ways in the embodiments of the present application respectively. The embodiment of the present application does not limit this and will not be described in detail.

[0053] In summary, the weight of the edge segmentation area is determined according to the values ​​of each pixel point in the intersection area of ​​the object segmentation contour area and the hair segmentation area; the object segmentation contour area and the edge segmentation area are fused according to the weight of the edge segmentation area to generate the target object segmentation area. Therefore, the object segmentation contour area and the edge segmentation area are fused according to the weight of the edge segmentation area to achieve refined object segmentation.

[0054] In order to accurately determine the object segmentation contour area and the hair segmentation area from the image to be processed, Figure 4 As shown, Figure 4 1 is a flow chart of another image processing method according to an exemplary embodiment. In the disclosed embodiment, the first image segmentation network includes an object segmentation network and a hair segmentation network, and the semantic feature map can be input into the object segmentation network and the hair segmentation network respectively to determine the object segmentation contour area and the hair segmentation area from the image to be processed. Figure 4 The illustrated embodiment may include the following steps:

[0055] Step 401: Obtain an image to be processed.

[0056] Step 402: extract semantic features from the image to be processed to obtain a semantic feature map of the image to be processed.

[0057] In the disclosed embodiment, the image to be processed can be input into the object segmentation network of the first segmentation network, and the encoding layer (e.g., Backbone encoder) of the object segmentation network can perform feature extraction on the image to be processed to obtain a feature image of the image to be processed. Furthermore, in order to reduce the amount of related calculations and increase the global receptive field of the feature image, the feature image of the image to be processed can be downsampled. For example, if the size of the image to be processed is 640×480, in order to make the feature image have a global receptive field, the feature image of the image to be processed can be downsampled 5 times, each time downsampling by 2 times, for a total of 32 times.

[0058] Furthermore, the feature image extracted from the image to be processed can be input into the object segmentation network of the first segmentation network, and the decoding layer (e.g., LR Branch decoder) of the object segmentation network can output the semantic feature map of the image to be processed. In order to amplify the semantic features, the feature image extracted from the image to be processed can be upsampled. For example, the feature image extracted from the image to be processed can be upsampled by 2 times, 2 times each time, and a total of 4 times. It should be noted that the first image segmentation network may include an object segmentation network and a hair segmentation network. For example, the first image segmentation network may include a Backbone encoder, an LR Branch decoder, and a Semantic Branch decoder. The Backbone encoder, the LR Branch decoder, and the Semantic Branch decoder can be connected in series to form an object segmentation network and a hair segmentation network.

[0059] Step 403: input the semantic feature map into the object segmentation network to obtain the object segmentation contour area output by the object segmentation network.

[0060] In an embodiment of the present disclosure, a semantic feature map may be input into an object segmentation network, and a decoding layer (eg, a Semantic Branch decoder) of the object segmentation network may output an object segmentation contour region.

[0061] Step 404, inputting the semantic feature map into the hair segmentation network to obtain the hair segmentation region output by the hair segmentation network.

[0062] In the disclosed embodiment, the semantic feature map may be input into a hair segmentation network, and the decoding layer of the hair segmentation network (eg, a Semantic Branch decoder) may output a hair segmentation region.

[0063] Step 405, scaling the image to be processed, and inputting the scaled image to be processed and the semantic feature map into a second image segmentation network to determine an edge segmentation region from the image to be processed.

[0064] It should be noted that, in order to make the size of the output object segmentation region and the hair segmentation region consistent with the size of the image to be processed, before the object segmentation network outputs the object segmentation region and the hair segmentation network outputs the hair segmentation region, if the size of the semantic feature map is inconsistent with the size of the image to be processed, the semantic feature map can be upsampled to make the size of the semantic feature map consistent with the size of the image to be processed. For example, the semantic feature map can be upsampled three times step by step, upsampled by 2 times each time, and upsampled by 8 times in total.

[0065] Step 406: According to the object segmentation contour area and the hair segmentation area, the object segmentation contour area and the edge segmentation area are fused to determine the target object segmentation area from the image to be processed.

[0066] In the embodiment of the present application, steps 401, 405-406 can be implemented in any way in the embodiments of the present application respectively, and the embodiment of the present application does not limit this and will not be repeated.

[0067] In summary, by inputting the semantic feature map into the object segmentation network, the object segmentation network outputs the object segmentation contour area; the semantic feature map is input into the hair segmentation network, and the hair segmentation network outputs the hair segmentation area. Therefore, by inputting the semantic feature map into the first image segmentation network, the object segmentation contour area and the hair segmentation area can be accurately determined from the image to be processed.

[0068] In order to accurately determine the edge segmentation area from the image to be processed, such as Figure 5 As shown, Figure 5 is a flow chart of another image processing method according to an exemplary embodiment. In the disclosed embodiment, the image to be processed may be scaled, and the scaled image to be processed and the semantic feature map are input into a second image segmentation network to determine an edge segmentation region from the image to be processed. Figure 5 The illustrated embodiment may include the following steps:

[0069] Step 501: Obtain an image to be processed.

[0070] Step 502: extract semantic features from the image to be processed to obtain a semantic feature map of the image to be processed.

[0071] Step 503: input the semantic feature map into the first image segmentation network to determine the object segmentation contour area and the hair segmentation area from the image to be processed.

[0072] Step 504 : determining a first scaled image and a second scaled image of the image to be processed according to the first set scaled ratio and the second set scaled ratio, wherein the first set scaled ratio is greater than the second set scaled ratio.

[0073] In the disclosed embodiment, the image to be processed may be multiplied by the first set zoom ratio to determine the first zoomed image of the image to be processed, and the image to be processed may be multiplied by the second set zoom ratio to determine the second zoomed image of the image to be processed. It should be noted that the first set zoom ratio is greater than the second set zoom ratio. For example, the first zoom ratio is one-half, and the second zoom ratio is one-quarter.

[0074] Step 505: Perform convolution processing on the first scaled image to obtain a feature map corresponding to the first scaled image.

[0075] Furthermore, the first scaled image may be subjected to a convolution operation to obtain a feature map corresponding to the first scaled image. For example, the first scaled image may be input into a convolutional neural network, and the convolutional neural network may output a feature map corresponding to the first scaled image.

[0076] Step 506: Fusing the feature map corresponding to the first scaled image, the second scaled image, and the semantic feature map to obtain a fused feature map.

[0077] Then, the feature map corresponding to the first zoomed image, the second zoomed image and the semantic feature map are subjected to feature fusion by an image fusion method to obtain a fused feature map. The image fusion method may be, for example, a pixel-level image fusion method.

[0078] Step 507: input the fused feature map to the second image segmentation network, and the second image segmentation network outputs the edge segmentation area.

[0079] In an embodiment of the present disclosure, the fused feature map may be input into a second image segmentation network, and a decoding layer of the second segmentation network may output an edge segmentation region.

[0080] Step 508: According to the object segmentation contour area and the hair segmentation area, the object segmentation contour area and the edge segmentation area are fused to determine the target object segmentation area from the image to be processed.

[0081] In the embodiment of the present application, steps 501-503 and 508 can be implemented in any way in the embodiments of the present application respectively. The embodiment of the present application does not limit this and will not be repeated.

[0082] For example, Figure 6As shown, taking the image to be processed as a portrait to be processed as an example, the portrait to be processed can be input into the backbone network Backbone encoder of the object segmentation network of the first segmentation network to extract features to obtain the feature image of the image to be processed. Then, the feature image of the image to be processed is input into the decoding layer LRBranch of the object segmentation network, and LR Branch can output the semantic feature map Semantic of the image to be processed. Then, the semantic feature map Semantic is input into the Semantic Branch of the first image segmentation network to determine the portrait segmentation contour area and the hair segmentation area from the portrait to be processed. At the same time, the image to be processed is scaled by half and a quarter to obtain the first scaled image and the second scaled image. The first scaled image is convolved to obtain the feature image corresponding to the first scaled image. The feature image corresponding to the first scaled image, the second scaled image and the semantic feature map Semantic are simultaneously input into the decoding layer of the Alpha Branch, and the decoding layer of the Alpha Branch can output the edge segmentation area. Then, according to the portrait segmentation contour area, the hair segmentation area and the edge segmentation area, a refined portrait segmentation result can be generated.

[0083] In summary, the first scaled image and the second scaled image of the image to be processed are determined according to the first set scaled ratio and the second set scaled ratio, wherein the first set scaled ratio is greater than the second set scaled ratio; convolution processing is performed on the first scaled image to obtain a feature map corresponding to the first scaled image; feature fusion is performed on the feature map corresponding to the first scaled image, the second scaled image, and the semantic feature map to obtain a fused feature map, and the fused feature map is input into the second image segmentation network, and the second image segmentation network outputs the edge segmentation area.

[0084] The image processing method of the disclosed embodiment inputs the semantic feature map of the image to be processed into the first image segmentation network to determine the object segmentation contour area and the hair segmentation area from the image to be processed, and at the same time, performs scaling processing on the image to be processed, and inputs the scaled image to be processed and the semantic feature map into the second image segmentation network to determine the edge segmentation area from the image to be processed; and then, the target object segmentation area is determined according to the object segmentation contour area, the hair segmentation area and the edge segmentation area. Thus, without the need for additional guide images, clear hair edges can be obtained, the hair edge sharpness and hair segmentation effect are improved, the feathering effect is reduced, and refined object segmentation is achieved.

[0085] In order to implement the above embodiment, the present disclosure provides an image processing device.

[0086] Figure 7FIG. 1 is a block diagram of an image processing device according to an exemplary embodiment. Figure 7 The image processing device includes: a first acquisition module 710, a second acquisition module 720, a first determination module 730, a second determination module 740 and a third determination module 750.

[0087] Among them, the first acquisition module 710 is configured to execute acquisition of the image to be processed; the second acquisition module 720 is configured to execute semantic feature extraction on the image to be processed to obtain the semantic feature map of the image to be processed; the first determination module 730 is configured to execute input of the semantic feature map into the first image segmentation network to determine the object segmentation contour area and the hair segmentation area from the image to be processed; the second determination module 740 is configured to execute scaling of the image to be processed, and input the scaled image to be processed and the semantic feature map into the second image segmentation network to determine the edge segmentation area from the image to be processed; the third determination module 750 is configured to execute fusion processing of the object segmentation contour area and the edge segmentation area according to the object segmentation contour area and the hair segmentation area to determine the target object segmentation area from the image to be processed.

[0088] As a possible implementation method of the embodiment of the present disclosure, the third determination module 750 is configured to perform: determining the weight of the edge segmentation area based on the values ​​of each pixel point in the intersection area of ​​the object segmentation contour area and the hair segmentation area; generating the target object segmentation area based on the object segmentation contour area, the edge segmentation area and the weight of the edge segmentation area.

[0089] As a possible implementation method of the embodiment of the present disclosure, the third determination module 750 is configured to execute: corroding the object segmentation contour area to obtain the processed object segmentation contour area; dilating the hair segmentation area to obtain the dilated hair segmentation area; blurring the processed object segmentation contour area and the processed hair segmentation area respectively to obtain the blurred object segmentation contour area and the blurred hair segmentation area; negating the blurred object segmentation contour area to obtain the negated object segmentation contour area; determining the intersection area of ​​the negated object segmentation contour area and the blurred hair segmentation area, and using the value of each pixel point in the intersection area as the weight of the edge segmentation area.

[0090] As a possible implementation method of an embodiment of the present disclosure, the first determination module is configured to execute: inputting the semantic feature map into the object segmentation network to obtain the object segmentation contour area output by the object segmentation network; inputting the semantic feature map into the hair segmentation network to obtain the hair segmentation area output by the hair segmentation network.

[0091] As a possible implementation method of an embodiment of the present disclosure, the second determination module is configured to execute: determining a first scaled image and a second scaled image of the image to be processed according to a first set scale ratio and a second set scale ratio, wherein the first set scale ratio is greater than the second set scale ratio; performing convolution processing on the first scaled image to obtain a feature map corresponding to the first scaled image; performing feature fusion on the feature map corresponding to the first scaled image, the second scaled image, and the semantic feature map to obtain a fused feature map; inputting the fused feature map into a second image segmentation network, and the second image segmentation network outputs an edge segmentation area.

[0092] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0093] The image processing device of the disclosed embodiment inputs the semantic feature map of the image to be processed into the first image segmentation network to determine the object segmentation contour area and the hair segmentation area from the image to be processed, and at the same time, performs scaling processing on the image to be processed, and inputs the scaled image to be processed and the semantic feature map into the second image segmentation network to determine the edge segmentation area from the image to be processed; and then, the target object segmentation area is determined according to the object segmentation contour area, the hair segmentation area and the edge segmentation area. Thus, without the need for additional guide images, a clear hair edge can be obtained, the hair edge sharpness and the hair segmentation effect are improved, the feathering effect is reduced, and refined object segmentation is achieved.

[0094] Figure 8 The invention is a block diagram of an electronic device for an image processing method according to an exemplary embodiment.

[0095] like Figure 8 As shown, the electronic device 800 includes:

[0096] The memory 810 and the processor 820, a bus 830 connecting different components (including the memory 810 and the processor 820), the memory 810 stores a computer program, and when the processor 820 executes the program, the image processing method described in the embodiment of the present disclosure is implemented.

[0097] Bus 830 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0098] The electronic device 800 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 800, including volatile and non-volatile media, removable and non-removable media.

[0099] The memory 810 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 840 and / or cache memory 850. The electronic device 800 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 860 may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 not shown, usually called a "hard drive"). Although Figure 8 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 830 via one or more data medium interfaces. The memory 810 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0100] A program / utility 880 having a set (at least one) of program modules 870 may be stored, for example, in the memory 810, such program modules 870 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 870 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0101] The electronic device 800 may also communicate with one or more external devices 890 (e.g., keyboard, pointing device, display 891, etc.), one or more devices that enable a user to interact with the electronic device 800, and / or any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 892. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 893. Figure 8 As shown, the network adapter 893 communicates with other modules of the electronic device 800 via the bus 830. It should be understood that although Figure 8Not shown, other hardware and / or software modules may be used in conjunction with electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0102] The processor 820 executes various functional applications and data processing by running the programs stored in the memory 810 .

[0103] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the image processing method of the embodiment of the present disclosure, and will not be repeated here.

[0104] The electronic device provided by the embodiment of the present disclosure can execute the image processing method as described above, by inputting the semantic feature map of the image to be processed into the first image segmentation network to determine the object segmentation contour area and the hair segmentation area from the image to be processed, and at the same time, scaling the image to be processed, and inputting the scaled image to be processed and the semantic feature map into the second image segmentation network to determine the edge segmentation area from the image to be processed; and then, according to the object segmentation contour area, the hair segmentation area and the edge segmentation area, the target object segmentation area is determined. Thus, without the need for additional guide images, a clear hair edge can be obtained, the hair edge sharpness and hair segmentation effect are improved, the feathering effect is reduced, and refined object segmentation is achieved.

[0105] In order to implement the above embodiments, the present disclosure also proposes a storage medium.

[0106] When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method as described above.

[0107] In order to implement the above embodiments, the present disclosure further provides a computer program product, including a computer program, which, when executed by a processor of an electronic device, enables the electronic device to perform the image processing method as described above.

[0108] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0109] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Get the image to be processed; Extracting semantic features of the image to be processed to obtain a semantic feature map of the image to be processed; Inputting the semantic feature map into a first image segmentation network to determine an object segmentation contour region and a hair segmentation region from the image to be processed; Scaling the image to be processed, and inputting the scaled image to be processed and the semantic feature map into a second image segmentation network to determine an edge segmentation region from the image to be processed; Determining the weight of the edge segmentation region according to the values ​​of each pixel point in the intersection area of ​​the object segmentation contour region and the hair segmentation region, and performing fusion processing on the object segmentation contour region and the edge segmentation region according to the weight of the edge segmentation region to determine the target object segmentation region from the image to be processed; Determining the weight of the edge segmentation area according to the values ​​of each pixel point in the intersection area of ​​the object segmentation contour area and the hair segmentation area includes: Performing an erosion process on the object segmentation contour region to obtain a processed object segmentation contour region; Performing expansion processing on the hair segmentation region to obtain the hair segmentation region after expansion processing; Performing blur processing on the processed object segmentation contour region and the processed hair segmentation region respectively, so as to obtain a blurred object segmentation contour region and a blurred hair segmentation region; Performing a negation operation on the blurred object segmentation contour area to obtain a negated object segmentation contour area; An intersection area between the inverted object segmentation contour area and the blurred hair segmentation area is determined, and the value of each pixel point in the intersection area is used as the weight of the edge segmentation area.

2. The method according to claim 1, characterized in that The first image segmentation network includes an object segmentation network and a hair segmentation network, and the inputting the semantic feature map into the first image segmentation network to determine the object segmentation contour area and the hair segmentation area from the image to be processed includes: Inputting the semantic feature map into the object segmentation network to obtain the object segmentation contour area output by the object segmentation network; The semantic feature map is input into the hair segmentation network to obtain the hair segmentation region output by the hair segmentation network.

3. The method according to claim 1, characterized in that Scaling the image to be processed and inputting the scaled image to be processed and the semantic feature map into a second image segmentation network to determine an edge segmentation region from the image to be processed includes: Determining a first scaled image and a second scaled image of the image to be processed according to a first set scale ratio and a second set scale ratio, wherein the first set scale ratio is greater than the second set scale ratio; Performing convolution processing on the first scaled image to obtain a feature map corresponding to the first scaled image; Performing feature fusion on the feature map corresponding to the first scaled image, the second scaled image, and the semantic feature map to obtain a fused feature map; The fused feature map is input into a second image segmentation network, and the second image segmentation network outputs an edge segmentation region.

4. An image processing device, characterized in that: include: A first acquisition module is configured to acquire an image to be processed; A second acquisition module is configured to perform semantic feature extraction on the image to be processed to obtain a semantic feature map of the image to be processed; A first determination module is configured to input the semantic feature map into a first image segmentation network to determine an object segmentation contour area and a hair segmentation area from the image to be processed; A second determination module is configured to perform scaling on the image to be processed, and input the scaled image to be processed and the semantic feature map into a second image segmentation network to determine an edge segmentation region from the image to be processed; A third determination module is configured to determine the weight of the edge segmentation area according to the values ​​of each pixel point in the intersection area of ​​the object segmentation contour area and the hair segmentation area, and perform fusion processing on the object segmentation contour area and the edge segmentation area according to the weight of the edge segmentation area to determine the target object segmentation area from the image to be processed; Determining the weight of the edge segmentation area according to the values ​​of each pixel point in the intersection area of ​​the object segmentation contour area and the hair segmentation area includes: Performing an erosion process on the object segmentation contour region to obtain a processed object segmentation contour region; Performing expansion processing on the hair segmentation region to obtain the hair segmentation region after expansion processing; Performing blur processing on the processed object segmentation contour region and the processed hair segmentation region respectively, so as to obtain a blurred object segmentation contour region and a blurred hair segmentation region; Performing a negation operation on the blurred object segmentation contour area to obtain a negated object segmentation contour area; An intersection area between the inverted object segmentation contour area and the blurred hair segmentation area is determined, and the value of each pixel point in the intersection area is used as the weight of the edge segmentation area.

5. The device according to claim 4, characterized in that The first image segmentation network includes an object segmentation network and a hair segmentation network, and the first determination module is configured to execute: Inputting the semantic feature map into the object segmentation network to obtain the object segmentation contour area output by the object segmentation network; The semantic feature map is input into the hair segmentation network to obtain the hair segmentation region output by the hair segmentation network.

6. The device according to claim 4, characterized in that The second determining module is configured to execute: Determining a first scaled image and a second scaled image of the image to be processed according to a first set scale ratio and a second set scale ratio, wherein the first set scale ratio is greater than the second set scale ratio; Performing convolution processing on the first scaled image to obtain a feature map corresponding to the first scaled image; Performing feature fusion on the feature map corresponding to the first scaled image, the second scaled image, and the semantic feature map to obtain a fused feature map; The fused feature map is input into a second image segmentation network, and the second image segmentation network outputs an edge segmentation region.

7. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing method according to any one of claims 1 to 3. 8 . A storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method according to claim 1 .

9. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor of an electronic device, enables the electronic device to execute the image processing method according to any one of claims 1 to 3.

Images