Image processing method, device, electronic device and storage medium
By identifying and processing the hair direction of the hair area and generating virtual hair and highlight maps, the problem of poor effect of the existing hair salon editing algorithm is solved, and a more natural and three-dimensional hair effect is achieved.
Patent Information
- Application Number
- CN202010617757.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-30
- Publication Date
- 2025-01-03
- Estimated Expiration
- 2040-06-30
AI Technical Summary
The existing hair salon photo editing algorithms are not effective when filtering variegated colors, resulting in poor hair salon photo editing.
By identifying the hair region in the target image, obtaining the hair direction of each pixel, and performing transition smoothing according to the hair direction, generating virtual hair and highlight maps that do not cross over and superimpose them into the image to enhance hair details and three-dimensional.
Effectively filter out variegated colors, supplement hair details, and improve the effect and texture of hair paintings.
Smart Images
Figure CN113870121B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to an image segmentation method, device, electronic device and storage medium. Background Art
[0002] The processing and enhancement of portrait photos has always been an important application in the field of computer vision and artificial intelligence. In recent years, the automatic beautification algorithm of digital images of people has made a series of progress, but it is mainly focused on the two major directions of facial features and human posture, such as beautification, whitening, acne removal, slimming and body shaping.
[0003] With the increase in user demand and the development of technology, methods for beautifying the hair of portraits have gradually emerged. Among them, the existing typical hair retouching algorithms mainly provide a "soft hair" function. Its core algorithm is to smooth the hair area of the portrait photo according to the direction of the hair to achieve the effect of filtering out noise and making the hair look smoother.
[0004] However, the "soft hair" function provided by the existing hair retouching algorithm has a relatively simple algorithm, which is essentially a smoothing operation on the hair area of the original image, and can only filter out some unsightly noise, resulting in poor hair retouching effect. Summary of the invention
[0005] In order to solve the technical problems described in the background technology, the embodiments of the present disclosure provide an image processing method, device, electronic device and storage medium. The technical solutions of the present disclosure are as follows:
[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, comprising:
[0007] Identify the hair region in the target image;
[0008] Obtaining the hair direction of each pixel in the hair area;
[0009] According to the hair direction, performing transition smoothing processing on the hair region to obtain an intermediate image;
[0010] In the hair region, non-intersecting virtual hair strands are generated along the hair strand direction, and the virtual hair strands are superimposed on the intermediate image to obtain a first image after processing the hair region of the target image.
[0011] Optionally, after performing transition smoothing processing on the hair region according to the hair direction to obtain an intermediate image, the method further includes:
[0012] In the hair region, a plurality of highlight maps spaced at a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the intermediate image to obtain a second image after processing the hair region of the target image.
[0013] Optionally, after superimposing the virtual hair into the intermediate image to obtain a first image after processing the hair region of the target image, the method further includes:
[0014] In the hair region, a plurality of highlight maps spaced apart by a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the first image.
[0015] Optionally, before generating non-intersecting virtual hair strands in the hair region along the hair strand direction, the method further includes:
[0016] In the hair region, a plurality of highlight maps spaced apart by a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the intermediate image to obtain a second image after processing the hair region of the target image;
[0017] The step of superimposing the virtual hair into the intermediate image comprises:
[0018] The virtual hair is superimposed on the second image.
[0019] Optionally, in the hair region, generating virtual hair strands that do not cross each other along the hair strand direction includes:
[0020] In the hair region, a plurality of sampling points are randomly generated at preset intervals;
[0021] Starting from the sampling point, the hair is extended along the hair direction until the boundary of the hair region is encountered, or when the hair is separated from the generated virtual hair by a second preset distance, the extension is stopped to obtain virtual hairs that do not cross each other.
[0022] Optionally, superimposing the virtual hair into the intermediate image includes:
[0023] The virtual hair is superimposed on the intermediate image by rendering.
[0024] Optionally, in the hair region, generating a plurality of highlight maps spaced at a first preset distance perpendicular to the hair direction includes:
[0025] In the hair area, along the hair direction, virtual highlight hair strands that do not cross each other and whose brightness is greater than or equal to a preset brightness value are generated;
[0026] The virtual highlight hair is Gaussian blurred perpendicular to the hair direction and at intervals of the first preset distance to obtain the highlight map.
[0027] Optionally, superimposing the highlight map into the intermediate image includes:
[0028] Adding the color value of the pixel of the highlight map to the color value of the target pixel in the intermediate image to obtain the adjusted color value of the target pixel;
[0029] The target pixel point is a pixel point in the intermediate image corresponding to a position of a pixel point of the highlight map.
[0030] Optionally, performing transition smoothing processing on the hair region according to the hair direction to obtain an intermediate image includes:
[0031] The hair region is subjected to transition smoothing processing along the hair direction and a direction perpendicular to the hair direction to obtain an intermediate image.
[0032] According to a second aspect of an embodiment of the present disclosure, an image processing device includes:
[0033] A hair region acquisition module is configured to identify a hair region in a target image;
[0034] A hair direction acquisition module is configured to acquire the hair direction of each pixel point in the hair area;
[0035] A smoothing processing module is configured to perform transition smoothing processing on the hair region according to the hair direction to obtain an intermediate image;
[0036] The detail supplement module is configured to generate non-intersecting virtual hair strands in the hair region along the hair strand direction, and superimpose the virtual hair strands into the intermediate image to obtain a first image after processing the hair region of the target image.
[0037] Optionally, the device further comprises:
[0038] The highlight processing module is configured to perform transition smoothing processing on the hair area according to the hair direction in the smoothing processing module, and after obtaining an intermediate image, generate a plurality of highlight maps spaced at a first preset distance in the hair area perpendicular to the hair direction, and superimpose the highlight maps on the intermediate image to obtain a second image after processing the hair area of the target image.
[0039] Optionally, the device further comprises:
[0040] The highlight processing module is configured to generate a plurality of highlight maps spaced by a first preset distance in the hair area, perpendicular to the hair direction, and superimpose the highlight maps on the first image after the detail supplement module superimposes the virtual hair into the intermediate image to obtain a first image after processing the hair area of the target image.
[0041] Optionally, the device further comprises:
[0042] The highlight processing module is configured to generate a plurality of highlight maps spaced by a first preset distance in the hair region perpendicular to the hair direction before the detail supplementing module generates non-intersecting virtual hairs in the hair region along the hair direction, and superimpose the highlight maps on the intermediate image to obtain a second image after processing the hair region of the target image;
[0043] When the detail supplement module superimposes the virtual hair into the intermediate image, the detail supplement module is specifically configured as follows:
[0044] The virtual hair is superimposed on the second image.
[0045] Optionally, the detail supplement module includes:
[0046] A sampling submodule is configured to randomly generate a plurality of sampling points in the hair region at preset intervals;
[0047] The extension submodule is configured to start from the sampling point and extend along the hair direction until it encounters the boundary of the hair area or stops extending when it is separated from the generated virtual hair by a second preset distance to obtain virtual hairs that do not cross each other.
[0048] Optionally, the detail supplement module further includes:
[0049] The rendering submodule is configured to superimpose the virtual hair into the intermediate image by rendering.
[0050] Optionally, the highlight processing module includes:
[0051] A virtual hair generation submodule is configured to generate virtual highlight hair in the hair region along the hair direction, which do not cross each other and whose brightness is greater than or equal to a preset brightness value;
[0052] The Gaussian blur submodule is configured to be perpendicular to the hair direction and to perform Gaussian blur on the virtual highlight hair at intervals of the first preset distance to obtain the highlight map.
[0053] Optionally, the highlight processing module includes:
[0054] The superposition submodule is configured to add the color value of the pixel of the highlight map to the color value of the target pixel in the intermediate image to serve as the adjusted color value of the target pixel;
[0055] The target pixel point is a pixel point in the intermediate image corresponding to a position of a pixel point of the highlight map.
[0056] Optionally, the smoothing processing module is specifically configured as follows:
[0057] The hair region is subjected to transition smoothing processing along the hair direction and a direction perpendicular to the hair direction to obtain an intermediate image.
[0058] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:
[0059] processor;
[0060] a memory for storing instructions executable by the processor;
[0061] Wherein, the processor is configured to execute the instructions to implement the above-mentioned image processing method.
[0062] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method described above.
[0063] According to a fifth aspect of an embodiment of the present disclosure, there is provided a computer program product comprising instructions, which, when executed on a computer, enables the computer to implement the above-mentioned image processing method.
[0064] Compared with the prior art, the present invention has the following advantages:
[0065] The technical solution provided by the embodiment of the present disclosure first identifies the hair area in the target image, then obtains the hair direction of each pixel in the hair area, and then performs smoothing processing on the hair area according to the hair direction to obtain an intermediate image, and then generates virtual hair that does not cross each other in the hair area along the hair direction, and superimposes the virtual hair into the intermediate image to obtain an image of the hair area of the target image after processing. Among them, the transition smoothing processing of the hair area can filter out the noise, and the superposition of the virtual hair can supplement the hair details. It can be seen that the technical solution provided by the embodiment of the present disclosure can not only filter out the noise in the hair area to achieve the effect of soft hair, but also further present clear hair details, thereby improving the effect of hairdressing and photo retouching.
[0066] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment;
[0068] Figure 2 is a flowchart of another image processing method according to an exemplary embodiment;
[0069] Figure 3 is a schematic diagram of a virtual hair in an exemplary embodiment;
[0070] Figure 4 is a schematic diagram of a specular map in an exemplary embodiment;
[0071] Figure 5 is a block diagram of an image processing device according to an exemplary embodiment;
[0072] Figure 6 is a block diagram of an electronic device according to an exemplary embodiment;
[0073] Figure 7 is a block diagram of another electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0074] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.
[0075] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0076] In order to solve the technical problem in the related art that the "soft hair" function provided by the hair retouching algorithm is simple and can only filter out some unsightly colors, thus resulting in poor hair modification effect, the embodiments of the present disclosure provide an image processing method, device, electronic device and storage medium.
[0077] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, such as Figure 1 As shown, the image processing method may include the following steps:
[0078] Step 101: Identify the hair region in the target image.
[0079] The target image is an image including a hair area, which may be an image stored locally by the electronic device or an image downloaded by the electronic device from a network, or any frame of an image in a video stored locally by the electronic device or any frame of an image in a video downloaded by the electronic device from a network.
[0080] In addition, the recognition of the hair area can use any method based on deep learning or neural network, or a non-deep learning method, or the hair area can be manually marked by the user. It should be noted that the recognition of the hair area is not limited to the method described here, and can also be any other method.
[0081] Step 102: Obtain the hair direction of each pixel in the hair area.
[0082] The hair direction may be obtained by using any one of the following methods: method 1, method 2, and method 3.
[0083] Method 1: A predetermined gradient operator is used to calculate the gradient of each pixel in the hair area, and then the hair direction of each pixel is calculated according to the gradient of each pixel.
[0084] Method 2: Pre-design filters in 180 discrete values from 0 to 179 degrees, and then convolve the neighborhood pattern of each pixel in the hair area of the target image with the filters along different directions to find the maximum value of the convolution result (i.e. find the maximum filter response). The direction corresponding to the maximum value is the hair direction of this pixel. The neighborhood pattern of a pixel refers to a square box centered on the pixel, which can contain 7x7 or 9x9 pixels.
[0085] Method 3: Manually annotate the hair directions in the hair area of a large number of sample images in advance, and use these data as input to train using machine learning and deep neural network methods to generate a model that can predict the hair direction. Then, the above-mentioned target image is input into the model, and the hair direction of each pixel point in the hair area of the target image can be output.
[0086] It should be noted that the acquisition of the hair direction is not limited to the above three methods, and may be any other method except the above three methods.
[0087] Step 103: performing transition smoothing processing on the hair region according to the hair direction to obtain an intermediate image.
[0088] Among them, transition smoothing can remove noise in the hair area and make the hair look smoother.
[0089] Step 104: in the hair region, non-intersecting virtual hair strands are generated along the hair strand direction, and the virtual hair strands are superimposed on the intermediate image to obtain an image after processing the hair region of the target image.
[0090] The virtual hair is hair added to the hair region of the target image in addition to the hair in the target image, and is used to supplement the hair details in the hair region of the target image. Therefore, when the virtual hair is superimposed on the intermediate image, the hair details can be supplemented when the transition smoothing process in step 103 processes the hair region too smoothly, thereby avoiding the loss of hair details. Specifically, for example, in the image processing method provided in the embodiment of the present disclosure, the generated virtual hair can be as follows: Figure 3 shown.
[0091] Optionally, after step 103, the following steps may also be included:
[0092] In the hair region, a plurality of highlight maps spaced at a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the intermediate image to obtain a second image after processing the hair region of the target image.
[0093] Among them, there is a preset distance between every two highlight maps, and the preset distance is pre-set. It should be noted here that the preset distance can also be modified according to the needs of the user. For example, an increase button and a decrease button can be set in the setting interface of the hairdressing photo editing. If a click operation on the increase button is detected, the current value of the preset distance is increased, and if a click operation on the decrease button is detected, the current value of the preset distance is decreased.
[0094] In addition, superimposing the highlight map onto the intermediate image can increase the three-dimensional sense of the hair area, thereby improving the overall texture of the image. Specifically, for example, in the image processing method provided in the embodiment of the present disclosure, the generated highlight map can be as follows: Figure 4 As shown. It should be noted that the highlight virtual hair can be rendered as white lines in the black background image, so that the highlight map can be obtained after Gaussian blurring the image. Therefore, the highlight map can be white, and the background where the highlight map is located is black. Figure 4 It is only a schematic diagram of the highlight map, and does not mean that the highlight map is black and the background is white in this embodiment.
[0095] Optionally, after superimposing the virtual hair into the intermediate image to obtain a first image after processing the hair region of the target image, the method further includes:
[0096] In the hair region, a plurality of highlight maps spaced apart by a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the first image.
[0097] Optionally, before generating non-intersecting virtual hair strands in the hair region along the hair strand direction, the method further includes:
[0098] In the hair region, a plurality of highlight maps spaced apart by a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the intermediate image to obtain a second image after processing the hair region of the target image;
[0099] The step of superimposing the virtual hair into the intermediate image comprises:
[0100] The virtual hair is superimposed on the second image.
[0101] The second image is the intermediate image superimposed with the virtual hair.
[0102] As can be seen from the above, the process of generating virtual hair and the process of generating a highlight map can be executed in parallel and superimposed on the intermediate image respectively to obtain the above-mentioned first image (i.e., the first image superimposed with the virtual hair) and the second image (i.e., the second image superimposed with the highlight map) respectively; the virtual hair can also be generated first, and then the virtual hair is superimposed on the intermediate image to obtain the first image, and then the highlight map is generated, and then the highlight map is superimposed on the first image; the highlight map can also be generated first, and then the highlight map is superimposed on the intermediate image to obtain the second image, and then the virtual hair is generated, and then the virtual hair is superimposed on the second image.
[0103] In addition, it should be noted that the process of generating virtual hair and the process of generating virtual hair can also exist separately. That is, only virtual hair can be generated, so that only virtual hair is superimposed in the intermediate image; or only highlight map can be generated, so that only highlight map is superimposed in the intermediate image.
[0104] In summary, the image processing method provided by the embodiment of the present disclosure first identifies the hair region in the target image, then obtains the hair direction of each pixel in the hair region, and then performs smoothing processing on the hair region according to the hair direction to obtain an intermediate image, and then generates virtual hair that does not cross each other in the hair region along the hair direction, and superimposes the virtual hair into the intermediate image, and / or, in the hair region, generates multiple highlight maps spaced at a first preset distance perpendicular to the hair direction, and superimposes the highlight map into the intermediate image to obtain an image after processing the hair region of the target image. Among them, performing transition smoothing processing on the hair region can filter out noise, superimposing virtual hair can supplement hair details, and superimposing highlight maps can increase the three-dimensional sense of the hair region, thereby improving the overall texture of the picture. It can be seen that the technical solution provided by the embodiment of the present disclosure can not only filter out the noise in the hair region to achieve the effect of soft hair, but also further present clear hair details and / or better three-dimensional sense, thereby improving the effect of hairdressing and photo retouching.
[0105] According to a second aspect of the embodiment of the present disclosure, another image processing method is provided, such as Figure 2 As shown, the image processing method may include the following steps:
[0106] Step 201: Identify the hair area in the target image.
[0107] The target image is an image including a hair area, which may be an image stored locally by the electronic device or an image downloaded by the electronic device from a network, or any frame of an image in a video stored locally by the electronic device or any frame of an image in a video downloaded by the electronic device from a network.
[0108] In addition, the recognition of the hair area can use any method based on deep learning or neural network, or a non-deep learning method, or the hair area can be manually marked by the user. It should be noted that the recognition of the hair area is not limited to the method described here, and can also be any other method.
[0109] Step 202: Obtain the hair direction of each pixel in the hair area.
[0110] The hair direction may be obtained by using any one of the following methods: method 1, method 2, and method 3.
[0111] Method 1: A predetermined gradient operator is used to calculate the gradient of each pixel in the hair area, and then the hair direction of each pixel is calculated according to the gradient of each pixel.
[0112] Method 2: Pre-design filters in 180 discrete values from 0 to 179 degrees, and then convolve the neighborhood pattern of each pixel in the hair area of the target image with the filters along different directions to find the maximum value of the convolution result (i.e. find the maximum filter response). The direction corresponding to the maximum value is the hair direction of this pixel. The neighborhood pattern of a pixel refers to a square box centered on the pixel, which can contain 7x7 or 9x9 pixels.
[0113] Method 3: Manually annotate the hair directions in the hair area of a large number of sample images in advance, and use these data as input to train using machine learning and deep neural network methods to generate a model that can predict the hair direction. Then, the above-mentioned target image is input into the model, and the hair direction of each pixel point in the hair area of the target image can be output.
[0114] It should be noted that the acquisition of the hair direction is not limited to the above three methods, and may be any other method except the above three methods.
[0115] Step 203: performing transition smoothing processing on the hair region along the hair direction and a direction perpendicular to the hair direction to obtain an intermediate image.
[0116] Among them, transition smoothing can remove noise in the hair area and make the hair look smoother.
[0117] In addition, the hair direction and the direction perpendicular to the hair direction both include a component of a first preset direction and a component of a second preset direction, and the first preset direction is perpendicular to the second preset direction. Specifically, the first preset direction is, for example, the positive direction of the x-axis of the coordinate system in the target image, and the second preset direction is the positive direction of the y-axis of the coordinate system. The x-component and y-component of the hair direction can be displayed in the image using different color channels, for example, red and green color channels can be used to correspond to the x-component and y-component sizes of the hair direction, respectively.
[0118] It can be seen that in the image processing method provided by the embodiment of the present disclosure, when the hair area is subjected to transition smoothing, it is performed in two directions: along the hair direction and in the direction perpendicular to the hair direction, that is, "two-dimensional" smoothing is performed, and the smoothing result is more blurred, which can more effectively remove noise.
[0119] Step 204: in the hair region, non-intersecting virtual hair strands are generated along the hair strand direction, and the virtual hair strands are superimposed on the intermediate image to obtain a first image after processing the hair region of the target image.
[0120] The virtual hair is hair added to the hair region of the target image in addition to the hair in the target image, and is used to supplement the hair details in the hair region of the target image. Therefore, when the virtual hair is superimposed on the intermediate image, the hair details can be supplemented when the transition smoothing process in step 203 processes the hair region too smoothly, thereby avoiding the loss of hair details. Specifically, for example, in the image processing method provided in the embodiment of the present disclosure, the generated virtual hair can be as follows: Figure 3 shown.
[0121] In addition, in the image processing method provided by the embodiment of the present disclosure, the generated virtual hair strands do not cross each other, so that the hairdressing and photo editing effect is better, thereby further improving the visual perception.
[0122] Optionally, in the hair region, generating virtual hair strands that do not cross each other along the hair strand direction includes:
[0123] In the hair region, a plurality of sampling points are randomly generated at preset intervals;
[0124] Starting from the sampling point, the hair is extended along the hair direction until the boundary of the hair region is encountered, or when the hair is separated from the generated virtual hair by a second preset distance, the extension is stopped to obtain virtual hairs that do not cross each other.
[0125] That is, in the image processing method provided by the embodiment of the present disclosure, when generating virtual hair, it is first necessary to randomly generate multiple sampling points at preset intervals in the hair area, for example, generate a1 to a50 sampling points, and then, start from these sampling points one by one and extend them along the hair direction as the generated virtual hair. In each extension process, it is terminated when encountering the boundary of the hair area or being too close to the generated virtual hair, so as to avoid the generated virtual hair from crossing each other.
[0126] Among them, since the above-mentioned sampling points are randomly generated, even if the same image is processed by the image processing method provided by the embodiment of the present disclosure, the positions of the generated sampling points may be different, resulting in different virtual hair extending from the sampling points, and thus different final hair retouching effects.
[0127] Optionally, superimposing the virtual hair into the intermediate image includes:
[0128] The virtual hair is superimposed on the intermediate image by rendering.
[0129] Among them, the virtual hair is superimposed on the above-mentioned intermediate image through rendering, so that the color, brightness, etc. of the virtual hair and the hair area in the intermediate image are more matched, that is, the virtual hair can be better integrated into the hair area in the intermediate image, thereby further improving the hairdressing and photo retouching effect.
[0130] Optionally, after step 203, the following steps may also be included:
[0131] In the hair region, a plurality of highlight maps spaced at a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the intermediate image to obtain a second image after processing the hair region of the target image.
[0132] Among them, there is a preset distance between every two highlight maps, and the preset distance is pre-set. It should be noted here that the preset distance can also be modified according to the needs of the user. For example, an increase button and a decrease button can be set in the setting interface of the hairdressing photo editing. If a click operation on the increase button is detected, the current value of the preset distance is increased, and if a click operation on the decrease button is detected, the current value of the preset distance is decreased.
[0133] In addition, superimposing the highlight map onto the intermediate image can increase the three-dimensional sense of the hair area, thereby improving the overall texture of the image. Specifically, for example, in the image processing method provided in the embodiment of the present disclosure, the generated highlight map can be as follows: Figure 4 shown.
[0134] In addition, in the image processing method provided in the embodiment of the present disclosure, a highlight map is generated perpendicular to the direction of the hair, so that the highlight map can be better matched with the hair area of the intermediate image, thereby achieving a better effect of hairdressing and photo retouching.
[0135] Optionally, after superimposing the virtual hair into the intermediate image to obtain a first image after processing the hair region of the target image, the method further includes:
[0136] In the hair region, a plurality of highlight maps spaced apart by a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the first image.
[0137] Optionally, before generating non-intersecting virtual hair strands in the hair region along the hair strand direction, the method further includes:
[0138] In the hair region, a plurality of highlight maps spaced apart by a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the intermediate image to obtain a second image after processing the hair region of the target image;
[0139] The step of superimposing the virtual hair into the intermediate image comprises:
[0140] The virtual hair is superimposed on the second image.
[0141] The second image is the intermediate image superimposed with the virtual hair.
[0142] As can be seen from the above, the process of generating virtual hair and the process of generating a highlight map can be executed in parallel and superimposed on the intermediate image respectively to obtain the above-mentioned first image (i.e., the first image superimposed with the virtual hair) and the second image (i.e., the second image superimposed with the highlight map) respectively; the virtual hair can also be generated first, and then the virtual hair is superimposed on the intermediate image to obtain the first image, and then the highlight map is generated, and then the highlight map is superimposed on the first image; the highlight map can also be generated first, and then the highlight map is superimposed on the intermediate image to obtain the second image, and then the virtual hair is generated, and then the virtual hair is superimposed on the second image.
[0143] In addition, it should be noted that the process of generating virtual hair and the process of generating virtual hair can also exist separately. That is, only virtual hair can be generated, so that only virtual hair is superimposed in the intermediate image; or only highlight map can be generated, so that only highlight map is superimposed in the intermediate image.
[0144] Optionally, in the hair region, generating a plurality of highlight maps spaced at a first preset distance perpendicular to the hair direction includes:
[0145] In the hair area, along the hair direction, virtual highlight hair strands that do not cross each other and whose brightness is greater than or equal to a preset brightness value are generated;
[0146] The virtual highlight hair is Gaussian blurred perpendicular to the hair direction and at intervals of the first preset distance to obtain the highlight map.
[0147] Wherein, the virtual highlight hair is generated in the hair area, and when Gaussian blur is performed only on the virtual highlight hair, the virtual highlight hair needs to be extracted from the hair area. Wherein, "the virtual highlight hair can be rendered as a line of a second color on a background image of a first color" to extract the virtual highlight hair from the hair area, and then the lines of the second color can be Gaussian blurred perpendicular to the direction of the hair and at intervals of a first preset distance to obtain a highlight map.
[0148] In addition, the first color may be black, and the second color may be white.
[0149] It can be seen that when generating a highlight map, it is necessary to process based on virtual hair. Therefore, when generating a highlight map, it is first necessary to generate virtual highlight hair that does not cross each other and has a brightness greater than or equal to a preset brightness value in the hair area along the hair direction, and then render the virtual highlight hair as a line of the second color on the background image of the first color, extract the virtual highlight hair from the hair area, and finally, perform Gaussian blur on the line along a direction perpendicular to the hair direction of the pixel point of the line and at intervals of a first preset distance, so that multiple highlight maps can be obtained. Among them, when generating a highlight map, the method for generating virtual hair based on can refer to the specific method for generating virtual hair in step 204, which will not be repeated here.
[0150] However, it should be noted that the specific process of generating virtual hair in the previous step 204 and the specific process of generating virtual hair in the process of generating the highlight map have slightly different parameters. Specifically, compared with the virtual hair generated in the previous step 204, the virtual hair generated in the process of generating the highlight map has a larger hair interval, a wider width, and a shorter length.
[0151] In addition, since the above-mentioned sampling points are randomly generated, the virtual hair generated based on the sampling points is random, and thus the highlight map obtained based on the virtual hair is also random. Therefore, even if the same image is processed using the image processing method provided by the embodiment of the present disclosure, the positions of the generated sampling points may be different, resulting in different virtual hair extending from the sampling points, and different highlight maps obtained based on the virtual hair, which in turn results in different final hairdressing and photo editing effects.
[0152] Optionally, superimposing the highlight map into the intermediate image includes:
[0153] Adding the color value of the pixel of the highlight map to the color value of the target pixel in the intermediate image to obtain the adjusted color value of the target pixel;
[0154] The target pixel point is a pixel point in the intermediate image corresponding to a position of a pixel point of the highlight map.
[0155] That is, the color value of each pixel of the specular map is added to the intermediate image to obtain the intermediate image superimposed with the specular map.
[0156] In summary, the image processing method provided by the embodiment of the present disclosure first identifies the hair region in the target image, then obtains the hair direction of each pixel in the hair region, and then performs smoothing processing on the hair region according to the hair direction to obtain an intermediate image, and then generates virtual hair that does not cross each other in the hair region along the hair direction, and superimposes the virtual hair into the intermediate image, and / or generates multiple highlight maps perpendicular to the hair direction and spaced by a first preset distance in the hair region, and superimposes the highlight map into the intermediate image to obtain an image after processing the hair region of the target image. Among them, performing transition smoothing processing on the hair region can filter out noise, superimposing virtual hair can supplement hair details, and superimposing highlight maps can increase the three-dimensional sense of the hair region, thereby improving the overall texture of the picture. It can be seen that the technical solution provided by the embodiment of the present disclosure can not only filter out the noise in the hair region to achieve the effect of soft hair, but also further present clear hair details and / or better three-dimensional sense, thereby improving the effect of hairdressing and photo retouching.
[0157] According to a third aspect of the embodiments of the present disclosure, there is provided an image processing apparatus, such as Figure 5 As shown, the image segmentation device 50 includes:
[0158] The hair region acquisition module 501 is configured to identify the hair region in the target image;
[0159] A hair direction acquisition module 502 is configured to acquire the hair direction of each pixel point in the hair area;
[0160] A smoothing processing module 503 is configured to perform transition smoothing processing on the hair region according to the hair direction to obtain an intermediate image;
[0161] The detail supplementation module 504 is configured to generate non-intersecting virtual hair strands in the hair region along the hair strand direction, and superimpose the virtual hair strands into the intermediate image to obtain a first image after processing the hair region of the target image.
[0162] Optionally, the device further comprises:
[0163] The highlight processing module 505 is configured to perform transition smoothing processing on the hair area according to the hair direction in the smoothing processing module 503, and after obtaining the intermediate image, generate a plurality of highlight maps spaced at a first preset distance in the hair area perpendicular to the hair direction, and superimpose the highlight maps on the intermediate image to obtain a second image after processing the hair area of the target image.
[0164] Optionally, the device further comprises:
[0165] The highlight processing module 505 is configured to, after the detail supplement module 504 superimposes the virtual hair into the intermediate image to obtain a first image after processing the hair area of the target image, generate a plurality of highlight maps spaced by a first preset distance in the hair area, perpendicular to the direction of the hair, and superimpose the highlight maps into the first image.
[0166] Optionally, the device further comprises:
[0167] The highlight processing module 505 is configured to generate a plurality of highlight maps spaced by a first preset distance in the hair region perpendicular to the hair direction before the detail supplementing module 504 generates non-intersecting virtual hairs in the hair region along the hair direction, and superimpose the highlight maps on the intermediate image to obtain a second image after processing the hair region of the target image;
[0168] When superimposing the virtual hair into the intermediate image, the detail supplementing module 504 is specifically configured as follows:
[0169] The virtual hair is superimposed on the second image.
[0170] Optionally, the detail supplement module 504 includes:
[0171] The sampling submodule 5041 is configured to randomly generate a plurality of sampling points in the hair region at preset intervals;
[0172] The extension submodule 5042 is configured to start from the sampling point and extend along the hair direction until it encounters the boundary of the hair area or stops extending when it is separated from the generated virtual hair by a second preset distance to obtain virtual hairs that do not cross each other.
[0173] Optionally, the detail supplement module 504 further includes:
[0174] The rendering submodule 5043 is configured to superimpose the virtual hair into the intermediate image by rendering.
[0175] Optionally, the highlight processing module 505 includes:
[0176] The virtual hair generation submodule 5051 is configured to generate virtual highlight hairs in the hair region along the hair direction, which do not cross each other and whose brightness is greater than or equal to a preset brightness value;
[0177] The Gaussian blur submodule 5052 is configured to be perpendicular to the hair direction and to perform Gaussian blur on the virtual highlight hair at intervals of the first preset distance to obtain the highlight map.
[0178] Optionally, the highlight processing module 505 includes:
[0179] The superposition submodule 5053 is configured to add the color value of the pixel of the highlight map to the color value of the target pixel in the intermediate image to serve as the adjusted color value of the target pixel;
[0180] The target pixel point is a pixel point in the intermediate image corresponding to a position of a pixel point of the highlight map.
[0181] Optionally, the smoothing processing module 503 is specifically configured to:
[0182] The hair region is subjected to transition smoothing processing along the hair direction and a direction perpendicular to the hair direction to obtain an intermediate image.
[0183] In summary, the image processing device provided by the embodiment of the present disclosure first identifies the hair region in the target image, then obtains the hair direction of each pixel in the hair region, and then performs smoothing processing on the hair region according to the hair direction to obtain an intermediate image, and then generates virtual hair that does not cross each other in the hair region along the hair direction, and superimposes the virtual hair into the intermediate image, and / or, in the hair region, generates multiple highlight maps spaced at a first preset distance perpendicular to the hair direction, and superimposes the highlight map into the intermediate image to obtain an image after processing the hair region of the target image. Among them, performing transition smoothing processing on the hair region can filter out noise, superimposing virtual hair can supplement hair details, and superimposing highlight maps can increase the three-dimensional sense of the hair region, thereby improving the overall texture of the picture. It can be seen that the technical solution provided by the embodiment of the present disclosure can not only filter out the noise in the hair region to achieve the effect of soft hair, but also further present clear hair details and / or better three-dimensional sense, thereby improving the effect of hairdressing and photo retouching.
[0184] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0185] Figure 6 is a block diagram of an electronic device according to an exemplary embodiment. Figure 6 , the terminal comprises:
[0186] Processor 610;
[0187] A memory 60 for storing instructions executable by the processor;
[0188] Wherein, the processor is configured to execute the instructions to implement the above-mentioned image processing method.
[0189] Figure 7 is a block diagram of another electronic device 700 according to an exemplary embodiment. For example, the device 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0190] Reference Figure 7 , the device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .
[0191] The processing component 702 generally controls the overall operation of the device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.
[0192] The memory 704 is configured to store various types of data to support operations on the device 700. Examples of such data include instructions for any application or method operating on the device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0193] The power supply component 706 provides power to the various components of the device 700. The power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 700.
[0194] The multimedia component 708 includes a screen that provides an output interface between the device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0195] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the device 700 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.
[0196] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0197] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the device 700. For example, the sensor assembly 714 can detect the open / closed state of the device 700, the relative positioning of components, such as the display and keypad of the device 700, and the sensor assembly 714 can also detect the position change of the device 700 or a component of the device 700, the presence or absence of user contact with the device 700, the orientation or acceleration / deceleration of the device 700, and the temperature change of the device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0198] The communication component 716 is configured to facilitate wired or wireless communication between the device 700 and other devices. The device 700 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 7G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0199] In an exemplary embodiment, the apparatus 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the image processing method described above.
[0200] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 707 including instructions, and the instructions can be executed by the processor 720 of the device 700 to complete the above-mentioned image processing method. Optionally, for example, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0201] According to another aspect of the implementation of the present disclosure, an embodiment of the present disclosure further provides a storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method described above.
[0202] According to another aspect of the embodiments of the present disclosure, a computer program product including instructions is provided, which enables the computer to implement the above-mentioned image processing method when the computer program product is executed on the computer.
[0203] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0204] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. An image processing method, characterized in that: include: Identify the hair region in the target image; Obtaining the hair direction of each pixel in the hair area; According to the hair direction, performing transition smoothing processing on the hair region to obtain an intermediate image; In the hair region, generating virtual hair strands that do not cross each other along the hair strand direction, and superimposing the virtual hair strands onto the intermediate image to obtain a first image after processing the hair region of the target image; Wherein, after performing transition smoothing processing on the hair region according to the hair direction to obtain an intermediate image, the method further includes: In the hair region, a plurality of highlight maps spaced apart by a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the intermediate image to obtain a second image after processing the hair region of the target image; Generating a plurality of highlight maps spaced by a first preset distance in the hair region perpendicular to the hair direction includes: In the hair area, along the hair direction, virtual highlight hair strands that do not cross each other and whose brightness is greater than or equal to a preset brightness value are generated; The virtual highlight hair is Gaussian blurred perpendicular to the hair direction and at intervals of the first preset distance to obtain the highlight map.
2. The image processing method according to claim 1, characterized in that: After superimposing the virtual hair into the intermediate image to obtain a first image after processing the hair region of the target image, the method further includes: In the hair region, a plurality of highlight maps spaced apart by a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the first image.
3. The image processing method according to claim 1, characterized in that: Before generating non-intersecting virtual hair strands in the hair region along the hair strand direction, the method further includes: In the hair region, a plurality of highlight maps spaced apart by a first preset distance are generated perpendicular to the hair direction, and the highlight maps are superimposed on the intermediate image to obtain a second image after processing the hair region of the target image; The step of superimposing the virtual hair into the intermediate image comprises: The virtual hair is superimposed on the second image.
4. The image processing method according to claim 1, characterized in that: In the hair region, non-intersecting virtual hair strands are generated along the hair strand direction, including: In the hair region, a plurality of sampling points are randomly generated at preset intervals; Starting from the sampling point, the hair is extended along the hair direction until the boundary of the hair region is encountered, or when the hair is separated from the generated virtual hair by a second preset distance, the extension is stopped to obtain virtual hairs that do not cross each other.
5. The image processing method according to claim 1, characterized in that: Superimposing the virtual hair into the intermediate image comprises: The virtual hair is superimposed on the intermediate image by rendering.
6. The image processing method according to claim 1, characterized in that: The specular map is superimposed onto the intermediate image, comprising: Adding the color value of the pixel of the highlight map to the color value of the target pixel in the intermediate image to obtain the adjusted color value of the target pixel; The target pixel point is a pixel point in the intermediate image corresponding to a position of a pixel point of the highlight map.
7. The image processing method according to claim 1, characterized in that: The step of performing transition smoothing processing on the hair region according to the hair direction to obtain an intermediate image comprises: The hair region is subjected to transition smoothing processing along the hair direction and a direction perpendicular to the hair direction to obtain an intermediate image.
8. An image processing device, characterized in that: include: A hair region acquisition module is configured to identify a hair region in a target image; A hair direction acquisition module is configured to acquire the hair direction of each pixel point in the hair area; A smoothing processing module is configured to perform transition smoothing processing on the hair region according to the hair direction to obtain an intermediate image; a detail supplementation module, configured to generate virtual hair strands that do not cross each other in the hair region along the hair strand direction, and superimpose the virtual hair strands into the intermediate image to obtain a first image after processing the hair region of the target image; a highlight processing module configured to perform transition smoothing processing on the hair region according to the hair direction in the smoothing processing module, and after obtaining an intermediate image, generate a plurality of highlight maps spaced by a first preset distance in the hair region perpendicular to the hair direction, and superimpose the highlight maps on the intermediate image to obtain a second image after processing the hair region of the target image; Among them, the highlight processing module also includes: A virtual hair generation submodule is configured to generate virtual highlight hair in the hair region along the hair direction, which do not cross each other and whose brightness is greater than or equal to a preset brightness value; The Gaussian blur submodule is configured to be perpendicular to the hair direction and to perform Gaussian blur on the virtual highlight hair at intervals of the first preset distance to obtain the highlight map.
9. The image processing device according to claim 8, characterized in that: The device also includes: The highlight processing module is configured to generate a plurality of highlight maps spaced by a first preset distance in the hair area, perpendicular to the hair direction, and superimpose the highlight maps on the first image after the detail supplement module superimposes the virtual hair into the intermediate image to obtain a first image after processing the hair area of the target image.
10. The image processing device according to claim 8, characterized in that: The device also includes: The highlight processing module is configured to generate a plurality of highlight maps spaced by a first preset distance in the hair region perpendicular to the hair direction before the detail supplementing module generates non-intersecting virtual hairs in the hair region along the hair direction, and superimpose the highlight maps on the intermediate image to obtain a second image after processing the hair region of the target image; When the detail supplement module superimposes the virtual hair into the intermediate image, the detail supplement module is specifically configured as follows: The virtual hair is superimposed on the second image.
11. The image processing device according to claim 8, characterized in that: The detail supplement module includes: A sampling submodule is configured to randomly generate a plurality of sampling points in the hair region at preset intervals; The extension submodule is configured to start from the sampling point and extend along the hair direction until it encounters the boundary of the hair area or stops extending when it is separated from the generated virtual hair by a second preset distance to obtain virtual hairs that do not cross each other.
12. The image processing device according to claim 8, characterized in that: The detail supplement module also includes: The rendering submodule is configured to superimpose the virtual hair into the intermediate image by rendering.
13. The image processing device according to claim 8, characterized in that: The highlight processing module includes: The superposition submodule is configured to add the color value of the pixel of the highlight map to the color value of the target pixel in the intermediate image to serve as the adjusted color value of the target pixel; The target pixel point is a pixel point in the intermediate image corresponding to a position of a pixel point of the highlight map.
14. The image processing device according to claim 8, characterized in that: The smoothing processing module is specifically configured as follows: The hair region is subjected to transition smoothing processing along the hair direction and a direction perpendicular to the hair direction to obtain an intermediate image.
15. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing method according to any one of claims 1 to 7.
16. A storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method as claimed in any one of claims 1 to 7.
17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.