Image processing method, device, electronic device and storage medium
By performing brightness suppression and enhancement processing on the facial area image and combining fusion processing technology, the problem of yellowing and dullness in the existing technology is solved, the image beautification effect is improved, and the natural improvement of facial skin quality is achieved.
Patent Information
- Application Number
- CN202210313147.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-28
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-03-28
AI Technical Summary
The prior art cannot effectively improve the yellow and dull skin condition of the face in the image, resulting in poor image beautification effect.
By acquiring the first brightness component of the pixels in the face area image in the preset color space, performing brightness suppression and enhancement processing, the second and third brightness components are obtained, and the fusion process is performed to replace the original first brightness component, so as to Improve the brightness of the highlighted areas on the face, while suppressing non-highlight areas and maintaining the three-dimensional sense of shadowed areas.
It effectively improves the yellow and dull skin condition on the face, improves the image beautification effect, makes the face brighter and whiter, and maintains a three-dimensional feeling.
Smart Images

Figure CN114972009B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art
[0002] With the development of Internet technology, beauty functions based on image processing have been widely used in a variety of applications. For example, in various live broadcast applications and short video applications, the blemishes on the face of the portrait can be located and then the blemishes can be smoothed to achieve image beautification effects.
[0003] In the related art, the image beautification function can only remove facial blemishes, but has no effective improvement on the yellow and dull skin condition of the face itself, resulting in poor image beautification effect in the related art. Summary of the invention
[0004] The present disclosure provides an image processing method, device, electronic device and storage medium to at least solve the problem in the related art that the yellow and dark skin condition of the face in the image cannot be effectively improved, resulting in poor image beautification effect. The technical solution of the present disclosure is as follows:
[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, including:
[0006] Acquire a facial region image, and determine a first brightness component of pixels in the facial region image in a preset color space;
[0007] Performing brightness suppression on the first brightness component of the pixel to obtain a second brightness component corresponding to the pixel; the second brightness component is smaller than the first brightness component, and the degree of brightness suppression is negatively correlated with the first brightness component;
[0008] Performing enhancement processing on the first brightness component of the pixel to obtain a third brightness component corresponding to the pixel;
[0009] Performing fusion processing on the first brightness component, the second brightness component and the third brightness component corresponding to the pixel to obtain a fused brightness component corresponding to the pixel;
[0010] The first brightness component of the pixel in the facial area image is replaced with the fused brightness component corresponding to the pixel to obtain a target facial area image.
[0011] In an exemplary embodiment, the preset color space is a YCbCr color space; and the step of suppressing the brightness of the first brightness component of the pixel to obtain the second brightness component corresponding to the pixel includes:
[0012] Determine a brightness calculation value by taking the Y component of the pixel as a base and a preset value as an exponent; the preset value is greater than 1;
[0013] A second brightness component corresponding to the pixel is determined according to the brightness calculation value.
[0014] In an exemplary embodiment, the preset value includes a first preset value and a second preset value, and the first preset value is different from the second preset value;
[0015] The method of determining the brightness calculation value by taking the Y component of the pixel as the base and the preset value as the exponent includes:
[0016] Taking the Y component of the pixel as a base and taking the first preset value and the second preset value as exponents respectively, determining a first brightness calculation value and a second brightness calculation value;
[0017] The determining, according to the brightness calculation value, a second brightness component corresponding to the pixel includes:
[0018] According to a preset suppression coefficient, the first brightness calculation value and the second brightness calculation value are linearly fused to obtain a second brightness component corresponding to the pixel.
[0019] In an exemplary embodiment, the step of enhancing the first brightness component of the pixel to obtain a third brightness component corresponding to the pixel includes:
[0020] Performing curve stretching on the first brightness component of the pixel to obtain a stretched brightness component corresponding to the pixel;
[0021] According to a preset stretching coefficient, the first brightness component corresponding to the pixel and the stretched brightness component are linearly fused to obtain a third brightness component corresponding to the pixel.
[0022] In an exemplary embodiment, the fusing process of the first brightness component, the second brightness component and the third brightness component corresponding to the pixel includes:
[0023] Determine a first product of a first fusion coefficient and a first brightness component corresponding to the pixel, and a second product of a second fusion coefficient and a third brightness component corresponding to the pixel; the second fusion coefficient is the second brightness component corresponding to the pixel, and the sum of the first fusion coefficient and the second fusion coefficient is 1;
[0024] A sum of the first product and the second product is determined to obtain a fused brightness component corresponding to the pixel.
[0025] In an exemplary embodiment, acquiring a facial region image includes:
[0026] Acquire an image to be processed, perform facial region detection on the image to be processed, and obtain an initial facial region image;
[0027] The initial facial region image is reduced based on a preset ratio to obtain a first facial region image; the preset scaling ratio is a positive number less than 1;
[0028] Performing Gaussian blur processing on the first facial region image to obtain a second facial region image;
[0029] The second facial region image is scaled back to an original size to obtain a facial region image; the original size is the size of the initial facial region image.
[0030] In an exemplary embodiment, the method further comprises:
[0031] According to the key point information in the target facial region image, the preset makeup material is superimposed on the target facial region image.
[0032] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:
[0033] A first brightness component determination unit is configured to acquire a facial region image and determine a first brightness component of pixels in the facial region image in a preset color space;
[0034] a brightness suppression unit configured to suppress the brightness of the first brightness component of the pixel to obtain a second brightness component corresponding to the pixel; the second brightness component is smaller than the first brightness component, and the degree of brightness suppression is negatively correlated with the first brightness component;
[0035] a brightness enhancement unit, configured to perform enhancement processing on the first brightness component of the pixel to obtain a third brightness component corresponding to the pixel;
[0036] a brightness fusion unit, configured to perform fusion processing on the first brightness component, the second brightness component and the third brightness component corresponding to the pixel to obtain a fused brightness component corresponding to the pixel;
[0037] The replacement unit is configured to replace the first brightness component of the pixel in the facial area image with the fused brightness component corresponding to the pixel to obtain a target facial area image.
[0038] In an exemplary embodiment, the preset color space is a YCbCr color space; the brightness suppression unit includes:
[0039] A brightness calculation unit is configured to determine a brightness calculation value using the Y component of the pixel as a base and a preset value as an exponent; the preset value is greater than 1;
[0040] The second brightness component determining unit is configured to determine a second brightness component corresponding to the pixel according to the brightness calculation value.
[0041] In an exemplary embodiment, the preset value includes a first preset value and a second preset value, and the first preset value is different from the second preset value;
[0042] The brightness calculation unit is specifically configured to determine a first brightness calculation value and a second brightness calculation value by taking the Y component of the pixel as a base and taking the first preset value and the second preset value as exponents respectively;
[0043] The second brightness component determination unit is specifically configured to perform linear fusion of the first brightness calculation value and the second brightness calculation value according to a preset suppression coefficient to obtain a second brightness component corresponding to the pixel.
[0044] In an exemplary embodiment, the brightness enhancement unit includes:
[0045] a stretching unit, configured to perform curve stretching on the first brightness component of the pixel to obtain a stretched brightness component corresponding to the pixel;
[0046] The linear fusion unit is configured to perform linear fusion on the first brightness component corresponding to the pixel and the stretched brightness component according to a preset stretching coefficient to obtain a third brightness component corresponding to the pixel.
[0047] In an exemplary embodiment, the brightness fusion unit includes:
[0048] A first determining unit is configured to determine a first product of a first fusion coefficient and a first brightness component corresponding to the pixel, and a second product of a second fusion coefficient and a third brightness component corresponding to the pixel; the second fusion coefficient is the second brightness component corresponding to the pixel, and the sum of the first fusion coefficient and the second fusion coefficient is 1;
[0049] The second determining unit is configured to determine the sum of the first product and the second product to obtain a fused brightness component corresponding to the pixel.
[0050] In an exemplary embodiment, the first brightness component determining unit includes:
[0051] A facial region detection unit is configured to acquire an image to be processed, perform facial region detection on the image to be processed, and obtain an initial facial region image;
[0052] A first scaling unit is configured to perform a reduction process on the initial facial region image based on a preset ratio to obtain a first facial region image;
[0053] A Gaussian blur processing unit is configured to perform Gaussian blur processing on the first facial region image to obtain a second facial region image;
[0054] The second scaling unit is configured to scale the second facial region image back to an original size to obtain a facial region image; the original size is the size of the initial facial region image.
[0055] In an exemplary embodiment, the apparatus further comprises:
[0056] The material superimposing unit is configured to superimpose the preset makeup material onto the target facial area image according to the key point information in the target facial area image.
[0057] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:
[0058] processor;
[0059] a memory for storing instructions executable by the processor;
[0060] Wherein, the processor is configured to execute the instructions to implement the image processing method of the first aspect mentioned above.
[0061] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of the electronic device, the electronic device is enabled to perform the image processing method of the first aspect described above.
[0062] According to a fifth aspect of an embodiment of the present disclosure, there is provided a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image processing method of the first aspect is implemented.
[0063] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:
[0064] By determining the first brightness component of the pixel in the facial area image in a preset color space, and performing brightness suppression and enhancement processing on the first brightness component of the pixel respectively, the second brightness component and the third brightness component corresponding to the pixel are obtained, and then the first brightness component, the second brightness component and the third brightness component corresponding to the pixel are fused to obtain a fused brightness component, and the first brightness component of the pixel is replaced by the fused brightness component of the pixel to obtain an improved target facial area image, thereby achieving the goal of brightening the highlight area of the face while suppressing the non-highlight area, ensuring that the shadow area is not affected, and brightening without losing the three-dimensional sense of the facial area, effectively improving the yellow and dull skin condition of the face itself, and improving the image beautification effect.
[0065] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.
[0067] Figure 1 is a schematic diagram of an application environment of an image processing method according to an exemplary embodiment;
[0068] Figure 2 is a flowchart of an image processing method according to an exemplary embodiment;
[0069] Figure 3a is a comparative schematic diagram before and after feathering treatment according to an exemplary embodiment;
[0070] Figure 3b is a schematic diagram showing comparison before and after brightness suppression according to an exemplary embodiment;
[0071] Figure 4 is a flowchart of another image processing method according to an exemplary embodiment;
[0072] Figure 5 is a block diagram of an image processing device according to an exemplary embodiment;
[0073] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0074] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.
[0075] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0076] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.
[0077] See also Figure 1 , which is a schematic diagram of an application environment of an image processing method according to an exemplary embodiment, the application environment may include a terminal 110 and a server 120, and the terminal 110 and the server 120 may be connected via a wired network or a wireless network.
[0078] The terminal 110 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal 110 runs client software such as an application (Application, referred to as App) that provides image processing functions. The application may be an independent application or a subroutine in the application. Exemplarily, the application may include a video application, a live broadcast application, etc. The user of the terminal 110 may log in to the application using pre-registered user information, which may include an account number and a password.
[0079] The server 120 may be a server that provides background services for the application in the terminal 110. The server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.
[0080] The image processing method of the embodiment of the present disclosure may be executed by an electronic device, which may be a terminal or a server. The method may be executed by the terminal or the server alone, or by the terminal and the server in cooperation with each other.
[0081] Figure 2 is a flowchart of an image processing method according to an exemplary embodiment. Figure 2 As shown, the image processing method is applied to Figure 1 Taking the terminal in as an example, the following steps may be included.
[0082] In step S201, a facial region image is acquired, and a first brightness component of pixels in the facial region image in a preset color space is determined.
[0083] The facial region image may be an image corresponding to the facial region in the image to be processed, and the image to be processed may be an image captured in real time or a pre-stored image, or a frame image in a video. Therefore, obtaining the facial region image may be obtaining the image to be processed, and performing facial region detection on the image to be processed to segment the facial region image. The facial region detection may adopt any detection method in the prior art, such as the commonly used detection method based on GMM (Gaussian Mixed Model) and CNN (Convolutional Neural Network). The disclosed embodiment does not specifically limit the facial region detection method.
[0084] In an exemplary embodiment, the facial region image may be a human face image. In order to avoid the obvious gap in the boundary of the subsequent facial brightening process, the facial image may be feathered to make the facial edge transition natural. Based on this, the above-mentioned acquisition of the facial region image may include:
[0085] Get the image to be processed;
[0086] Performing facial region detection on the image to be processed to obtain an initial facial region image;
[0087] Reducing the initial facial region image based on a preset ratio to obtain a first facial region image;
[0088] Performing Gaussian blur processing on the first facial region image to obtain a second facial region image;
[0089] The second facial region image is scaled back to an original size to obtain a facial region image; the original size is the size of the initial facial region image.
[0090] Specifically, the preset ratio is a positive number less than 1, for example, the preset ratio is 1 / 4, if the initial facial region image is 512*512, then the first facial region image is 128*128. Among them, the reduction method can adopt a bilinear interpolation method or a bicubic interpolation method, and the specific bilinear interpolation or bicubic interpolation process can refer to the introduction in the relevant technology, which will not be repeated here.
[0091] When Gaussian blur processing is performed on the first facial region image, for each pixel in the first facial image, a weighted average of other pixels in the blur window can be taken. The larger the blur window, the stronger the blur effect, which is more conducive to edge smoothing. The weight of each pixel in the blur window can be determined based on a two-dimensional Gaussian function. For example, the blur window can be set to 11*11. Figure 3a The figure shows a comparison diagram of a facial region image before and after feathering processing, wherein the figure on the left is the facial region image before feathering processing, and the figure on the right is the facial region image after feathering processing.
[0092] In the above embodiment, by performing feathering processing on the facial region image, the facial edge transition is made more natural, and the boundary gap caused by the subsequent brightening processing is avoided, which is beneficial to improving the image processing effect.
[0093] In the embodiment of the present disclosure, when determining the first brightness component of pixels in a facial area image in a preset color space, the facial area image can be first converted from the original color space to the preset color space, and then the brightness component of each pixel in the preset color space, i.e., the first brightness component, is obtained.
[0094] Exemplarily, the original color space of the facial region image may be an RGB (red, green, blue) color space, and the preset color space may be a YCbCr color space, where Y is a brightness component, and Cb and Cr are chromaticity components. The facial region image is converted from the RGB color space to the YCbCr color space, and the brightness component of each pixel in the YCbCr color space, i.e., the first brightness component, can be obtained according to the Y channel value of the YCbCr color space.
[0095] It should be noted that the Y channel value of the YCbCr color space ranges from 0 to 1.
[0096] In step S203, brightness suppression is performed on the first brightness component of the pixel to obtain a second brightness component corresponding to the pixel.
[0097] The second brightness component is smaller than the first brightness component, and the degree of brightness suppression is negatively correlated with the first brightness component. The degree of brightness suppression refers to the ratio of the difference between the second brightness component and the first brightness component to the first brightness component. The degree of brightness suppression is negatively correlated with the first brightness component, which means that the larger the first brightness component, the smaller the degree of brightness suppression, and the smaller the first brightness component, the greater the degree of brightness suppression.
[0098] It should be noted that the above-mentioned pixel is any pixel in the facial region image, that is, for each pixel in the facial region image, the first brightness component of the pixel is brightness suppressed to obtain the second brightness component corresponding to the pixel. Similarly, the subsequent steps of the embodiment of the present disclosure are also for processing each pixel. Through the above-mentioned brightness suppression method, a highlight image highlighting the bright area can be obtained to ensure a three-dimensional effect. Figure 3b Shown is a schematic comparison diagram before and after brightness suppression provided by an embodiment of the present disclosure, wherein the left image is a face image before brightness suppression, and the right image is a face highlight image after brightness suppression. It can be seen that brightness suppression makes the face image more three-dimensional.
[0099] Taking the preset color space as the YCbCr color space as an example, the value range of the Y channel value (i.e., the first brightness component) of the YCbCr color space is 0 to 1. Then, in an exemplary implementation, brightness suppression is performed on the first brightness component of the pixel to obtain the second brightness component corresponding to the pixel, which may include:
[0100] Determine a brightness calculation value by taking the Y component of the pixel as a base and a preset value as an exponent;
[0101] A second brightness component corresponding to the pixel is determined according to the brightness calculation value.
[0102] Specifically, the preset value is greater than 1. The brightness calculation value may be used as the second brightness component of the corresponding pixel, or the second brightness component corresponding to the pixel may be further determined based on the brightness calculation value.
[0103] For example, the brightness calculation value can be expressed as y θ , where y is the Y component of the pixel, 0<y<1; θ is a preset value, θ>1. Taking θ=2 as an example, when y=0.9, the brightness calculation value is y 2 =0.81, when y=0.1, the brightness calculation value is y 2 = 0.01, if the calculated brightness value is directly used as the second brightness component of the pixel, it is obvious that the second brightness component is smaller than the first brightness component, and brightness suppression is achieved. When y = 0.9, When y = 0.1, That is, the degree of brightness suppression is negatively correlated with the first brightness component.
[0104] In the above embodiment, for the Y component of the YCbCr color space, considering that the value range of the Y component is 0 to 1, the brightness calculation value is determined by taking the Y component of the pixel as the base and the preset value as the exponent, and then the second brightness component of the pixel is determined based on the brightness calculation value. This can efficiently achieve brightness suppression and ensure that the degree of brightness suppression is negatively correlated with the first brightness component, which is conducive to improving the real-time processing of the image.
[0105] In an exemplary embodiment, in order to improve the brightness suppression effect and flexibility, the preset value may include a first preset value and a second preset value, and the first preset value and the second preset value are different. Obviously, the first preset value and the second preset value are both greater than 1. Then, when the Y component of the pixel is used as the base and the preset value is used as the exponent to determine the brightness calculation value, it may include: using the Y component of the pixel as the base and the first preset value and the second preset value as the exponent to determine the first brightness calculation value and the second brightness calculation value.
[0106] Correspondingly, when determining the second brightness component corresponding to the pixel according to the brightness calculation value, the above method may include: linearly fusing the first brightness calculation value and the second brightness calculation value according to a preset suppression coefficient to obtain the second brightness component corresponding to the pixel.
[0107] The preset suppression coefficient has a value range of 0 to 1, and can be set within the value range according to actual experience.
[0108] In the above implementation, the linear fusion formula of the first brightness calculation value and the second brightness calculation value according to the preset suppression coefficient can be expressed as:
[0109]
[0110] Wherein, Highlight represents the second brightness component of the pixel; θ1 is a first preset value, θ2 is a second preset value, θ1>1, θ2>1; is the first brightness calculation value of the pixel, is the second brightness calculation value of the pixel, 0<y<1; α is the preset suppression coefficient, 0<α<1. Obviously, when θ2<θ1 (for example, θ2=2, θ1=6), the smaller α is, the greater the brightness suppression can be achieved, and then by adjusting the preset suppression coefficient, different brightness suppression effects can be quickly achieved, which improves the flexibility of brightness suppression, and can obtain a better brightness suppression effect by setting the preset suppression coefficient to ensure the shadow area.
[0111] In step S205, the first brightness component of the pixel is enhanced to obtain a third brightness component corresponding to the pixel.
[0112] It can be understood that the enhancement processing can make the first brightness component of the pixel have an overall improvement, that is, the third brightness component is greater than the first brightness component.
[0113] In an exemplary implementation, the above step S205 may include:
[0114] Performing curve stretching on the first brightness component of the pixel to obtain a stretched brightness component corresponding to the pixel;
[0115] According to a preset stretching coefficient, the first brightness component corresponding to the pixel and the stretched brightness component are linearly fused to obtain a third brightness component corresponding to the pixel.
[0116] In a specific implementation, the first brightness component of the pixel can be stretched multiple times based on the color curve formula to obtain a better brightening effect. For example, taking two curve stretchings as an example, the brightness component after the first curve stretching can be expressed as y1=2*yy 2 , the brightness component after the second curve stretching can be expressed as y2 = 2*y1-y1 2 , then the stretched brightness component y′ corresponding to the pixel is y2, where y is the first brightness component of the pixel.
[0117] It can be understood that the color curve formula used in curve stretching is not limited to the color curve formula in the above example, and other color curve formulas, such as the gamma curve formula, etc., can also be used.
[0118] The preset stretching coefficient can be set according to the actual required degree of brightening. For example, the linear fusion formula of the first brightness component and the stretched brightness component corresponding to the pixel according to the preset stretching coefficient can be expressed as:
[0119]
[0120] in, Represents the third brightness component corresponding to the pixel; y is the first brightness component of the pixel; y′ is the stretched brightness component of the pixel; β is the preset stretching coefficient. In the above formula, the degree of brightening can be adjusted by adjusting β. The larger β is, the greater the brightening is.
[0121] In the above embodiment, the brightening effect can be quickly achieved by curve stretching processing and linear fusion of the first brightness component and the stretched brightness component corresponding to the pixel, which is conducive to the real-time processing of the image.
[0122] It should be noted that the execution order of step S203 and step S205 is not limited to the above-mentioned implementation mode. The brightness enhancement processing of step S205 can also be performed first, and then the brightness suppression processing of step S203 can be performed, or step S203 and step S205 can be performed simultaneously. The embodiments of the present disclosure do not make specific limitations on this.
[0123] In step S207, the first brightness component, the second brightness component and the third brightness component corresponding to the pixel are fused to obtain a fused brightness component corresponding to the pixel.
[0124] By performing the fusion processing of step 207 on each pixel, the highlight area of the face is brightened more and the shadow area is brightened less, thereby maintaining the original structure of the facial area without losing the three-dimensional sense of the facial area.
[0125] In an exemplary embodiment, the implementation of the above step S207 may include:
[0126] Determine a first product of a first fusion coefficient and a first brightness component corresponding to the pixel, and a second product of a second fusion coefficient and a third brightness component corresponding to the pixel; the second fusion coefficient is the second brightness component corresponding to the pixel, and the sum of the first fusion coefficient and the second fusion coefficient is 1;
[0127] A sum of the first product and the second product is determined to obtain a fused brightness component corresponding to the pixel.
[0128] The specific implementation can be expressed as the following linear fusion formula:
[0129]
[0130] Wherein, Mix represents the fused brightness component of the pixel; Highlight represents the second fusion coefficient, i.e., the second brightness component of the pixel; y represents the first brightness component of the pixel; is the third brightness component of the pixel; (1-Highlight) represents the first fusion coefficient.
[0131] In the above embodiment, by using the second brightness component corresponding to the pixel as the second fusion coefficient, and utilizing a simple linear fusion method in combination with the second fusion coefficient to fuse the first brightness component and the third brightness component corresponding to the pixel, it is beneficial to improve image processing efficiency and realize real-time processing of the image.
[0132] In step S209, the first brightness component of the pixel in the facial region image is replaced with the fused brightness component corresponding to the pixel to obtain a target facial region image.
[0133] In a specific implementation, the first brightness component of each pixel in the preset color space may be replaced by the fused brightness component corresponding to the pixel, and then each pixel may be converted from the preset color space back to the original color space, thereby obtaining a processed target facial area image.
[0134] Taking the preset color space as YCbCr color space and the original color space of the facial area image as RGB color space as an example, after obtaining the fused brightness component corresponding to each pixel, the Y channel value of each pixel is replaced with the corresponding fused brightness component and then each pixel is converted from the YCbCr color space back to the RGB color space to obtain the processed target facial area image.
[0135] The disclosed embodiment can suppress non-highlight areas while focusing on brightening the highlight areas of the face, ensuring that the shadow areas are not affected, and brightening without losing the three-dimensional sense of the facial area. It effectively improves the yellow and dull skin condition of the face, making the face translucent and fair, with a creamy and delicate skin texture, thereby improving the image beautification effect. In addition, the processing method is simple and real-time processing of images can be achieved.
[0136] In an exemplary embodiment, in order to further improve the visual effect of image beautification, as Figure 4 A flowchart of another image processing method is provided, and the method may also include:
[0137] In step S211, preset makeup materials are superimposed on the target facial region image according to key point information in the target facial region image.
[0138] In a specific implementation, key point detection may be performed on the target facial region image to obtain key point information.
[0139] Among them, key point detection can use a pre-trained key point detection model, input the target facial area image into the key point detection model for key point detection, and obtain output key point information. The key point information can include the position coordinates of each key point, and the position coordinates of each key point can locate the facial contour, eyes, eyebrows, lips, and nose contour in the target facial area image.
[0140] Then, the preset makeup material is obtained. Specifically, the preset makeup material can be obtained from the makeup material library, and a default makeup material can be provided, or a selected makeup material can be obtained according to a user's selection operation of the makeup material. For example, the preset makeup material can include blush, lipstick, eye shadow, and other materials used to modify facial skin color.
[0141] Furthermore, the preset makeup material is superimposed on the target facial area image according to the above key point information.
[0142] Specifically, each facial sub-region can be determined according to the key point information, and then the corresponding preset makeup material is superimposed on the facial sub-region according to the size of the corresponding facial sub-region, thereby improving the facial complexion and further improving the visual effect of image beautification.
[0143] Figure 5 is a block diagram of an image processing device according to an exemplary embodiment. Figure 5 , the image processing device 500 comprises:
[0144] The first brightness component determination unit 510 is configured to acquire a facial region image and determine a first brightness component of pixels in the facial region image in a preset color space;
[0145] A brightness suppression unit 520 is configured to perform brightness suppression on the first brightness component of the pixel to obtain a second brightness component corresponding to the pixel; the second brightness component is smaller than the first brightness component, and the degree of brightness suppression is negatively correlated with the first brightness component;
[0146] The brightness enhancement unit 530 is configured to perform enhancement processing on the first brightness component of the pixel to obtain a third brightness component corresponding to the pixel;
[0147] A brightness fusion unit 540 is configured to perform fusion processing on the first brightness component, the second brightness component and the third brightness component corresponding to the pixel to obtain a fused brightness component corresponding to the pixel;
[0148] The replacement unit 550 is configured to replace the first brightness component of the pixel in the facial area image with the fused brightness component corresponding to the pixel to obtain a target facial area image.
[0149] In an exemplary embodiment, the preset color space is a YCbCr color space; the brightness suppression unit 520 includes:
[0150] A brightness calculation unit is configured to determine a brightness calculation value using the Y component of the pixel as a base and a preset value as an exponent; the preset value is greater than 1;
[0151] The second brightness component determining unit is configured to determine a second brightness component corresponding to the pixel according to the brightness calculation value.
[0152] In an exemplary embodiment, the preset value includes a first preset value and a second preset value, and the first preset value is different from the second preset value;
[0153] The brightness calculation unit is specifically configured to determine a first brightness calculation value and a second brightness calculation value by taking the Y component of the pixel as a base and taking the first preset value and the second preset value as exponents respectively;
[0154] The second brightness component determination unit is specifically configured to perform linear fusion of the first brightness calculation value and the second brightness calculation value according to a preset suppression coefficient to obtain a second brightness component corresponding to the pixel.
[0155] In an exemplary embodiment, the brightness enhancement unit 530 includes:
[0156] a stretching unit, configured to perform curve stretching on the first brightness component of the pixel to obtain a stretched brightness component corresponding to the pixel;
[0157] The linear fusion unit is configured to perform linear fusion on the first brightness component corresponding to the pixel and the stretched brightness component according to a preset stretching coefficient to obtain a third brightness component corresponding to the pixel.
[0158] In an exemplary embodiment, the brightness fusion unit 540 includes:
[0159] A first determining unit is configured to determine a first product of a first fusion coefficient and a first brightness component corresponding to the pixel, and a second product of a second fusion coefficient and a third brightness component corresponding to the pixel; the second fusion coefficient is the second brightness component corresponding to the pixel, and the sum of the first fusion coefficient and the second fusion coefficient is 1;
[0160] The second determining unit is configured to determine the sum of the first product and the second product to obtain a fused brightness component corresponding to the pixel.
[0161] In an exemplary embodiment, the first brightness component determining unit 510 includes:
[0162] A facial region detection unit is configured to acquire an image to be processed, perform facial region detection on the image to be processed, and obtain an initial facial region image;
[0163] A first scaling unit is configured to perform a reduction process on the initial facial region image based on a preset ratio to obtain a first facial region image;
[0164] A Gaussian blur processing unit is configured to perform Gaussian blur processing on the first facial region image to obtain a second facial region image;
[0165] The second scaling unit is configured to scale the second facial region image back to an original size to obtain a facial region image; the original size is the size of the initial facial region image.
[0166] In an exemplary embodiment, the apparatus further comprises:
[0167] The material superimposing unit is configured to superimpose the preset makeup material onto the target facial area image according to the key point information in the target facial area image.
[0168] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0169] In an exemplary embodiment, an electronic device is also provided, including a processor; a memory for storing instructions executable by the processor; wherein, when the processor is configured to execute the instructions stored in the memory, it implements any one of the image processing methods provided in the embodiments of the present disclosure.
[0170] The electronic device may be a terminal, a server or a similar computing device. For example, the electronic device is a terminal. Figure 6 is a block diagram of a terminal for image processing according to an exemplary embodiment, specifically:
[0171] The terminal may include components such as an RF (Radio Frequency) circuit 610, a memory 620 including one or more computer-readable storage media, an input unit 630, a display unit 640, a sensor 650, an audio circuit 660, a WiFi (wireless fidelity) module 670, a processor 680 including one or more processing cores, and a power supply 690. Those skilled in the art will appreciate that Figure 6 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0172] The RF circuit 610 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is handed over to one or more processors 680 for processing; in addition, the data related to the uplink is sent to the base station. Generally, the RF circuit 610 includes but is not limited to an antenna, at least one amplifier, a tuner, one or more oscillators, a user identity module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, etc. In addition, the RF circuit 610 can also communicate with the network and other terminals through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), etc.
[0173] The memory 620 can be used to store software programs and modules. The processor 680 executes various functional applications and data processing by running the software programs and modules stored in the memory 620. The memory 620 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, application programs required for functions, etc.; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 620 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 620 may also include a memory controller to provide the processor 680 and the input unit 630 with access to the memory 620.
[0174] The input unit 630 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit 630 may include a touch-sensitive surface 631 and other input devices 632. The touch-sensitive surface 631, also known as a touch display screen or touchpad, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch-sensitive surface 631), and drive corresponding connection devices according to a pre-set program. Optionally, the touch-sensitive surface 631 may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 680, and can receive and execute commands sent by the processor 680. In addition, the touch-sensitive surface 631 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 631, the input unit 630 may further include other input devices 632. Specifically, the other input devices 632 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.
[0175] The display unit 640 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the terminal, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit 640 may include a display panel 641. Optionally, the display panel 641 may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface 631 may cover the display panel 641. When the touch-sensitive surface 631 detects a touch operation on or near it, it is transmitted to the processor 680 to determine the type of touch event, and then the processor 680 provides corresponding visual output on the display panel 641 according to the type of touch event. Among them, the touch-sensitive surface 631 and the display panel 641 can be two independent components to realize input and output functions, but in some embodiments, the touch-sensitive surface 631 and the display panel 641 can also be integrated to realize input and output functions.
[0176] The terminal may also include at least one sensor 650, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 641 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 641 and / or the backlight when the terminal is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the terminal posture (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can be configured in the terminal, they will not be repeated here.
[0177] The audio circuit 660, the speaker 661, and the microphone 662 can provide an audio interface between the user and the terminal. The audio circuit 660 can transmit the received audio data to the speaker 661 after converting the electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 662 converts the collected sound signal into an electrical signal, which is received by the audio circuit 660 and converted into audio data, and then the audio data is processed by the output processor 680, and then sent to another terminal through the RF circuit 610, or the audio data is output to the memory 620 for further processing. The audio circuit 660 may also include an earplug jack to provide communication between an external headset and the terminal.
[0178] WiFi is a short-range wireless transmission technology. The terminal can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 670, which provides users with wireless broadband Internet access. Figure 6 A WiFi module 670 is shown, but it is understandable that it is not an essential component of the terminal and can be omitted as needed without changing the essence of the invention.
[0179] The processor 680 is the control center of the terminal, and uses various interfaces and lines to connect various parts of the entire terminal. By running or executing software programs and / or modules stored in the memory 620, and calling data stored in the memory 620, the processor 680 performs various functions of the terminal and processes data, thereby monitoring the terminal as a whole. Optionally, the processor 680 may include one or more processing cores; preferably, the processor 680 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 680.
[0180] The terminal also includes a power supply 690 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 680 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 690 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.
[0181] Although not shown, the terminal may also include a camera, a Bluetooth module, etc., which will not be described in detail herein. Specifically in this embodiment, the terminal also includes a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for executing the image processing method provided in the above method embodiment.
[0182] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 620 including instructions, and the instructions can be executed by a processor 680 of the device 600 to complete the above-mentioned image processing method. Alternatively, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0183] In an exemplary embodiment, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, any one of the image processing methods provided in the embodiments of the present disclosure is implemented.
[0184] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0185] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. An image processing method, characterized in that: include: Acquire a facial region image, and determine a first brightness component of pixels in the facial region image in a preset color space; Performing brightness suppression on the first brightness component of the pixel to obtain a second brightness component corresponding to the pixel; The second brightness component is smaller than the first brightness component, and the degree of brightness suppression is negatively correlated with the first brightness component; Performing enhancement processing on the first brightness component of the pixel to obtain a third brightness component corresponding to the pixel; Determine a first product of a first fusion coefficient and a first brightness component corresponding to the pixel, and a second product of a second fusion coefficient and a third brightness component corresponding to the pixel; The second fusion coefficient is a second brightness component corresponding to the pixel, and the sum of the first fusion coefficient and the second fusion coefficient is 1; Determine the sum of the first product and the second product to obtain a fused brightness component corresponding to the pixel; The first brightness component of the pixel in the facial area image is replaced with the fused brightness component corresponding to the pixel to obtain a target facial area image.
2. The image processing method according to claim 1, characterized in that: The preset color space is the YCbCr color space; The step of suppressing the brightness of the first brightness component of the pixel to obtain a second brightness component corresponding to the pixel includes: Determine a brightness calculation value by taking the Y component of the pixel as a base and a preset value as an exponent; the preset value is greater than 1; A second brightness component corresponding to the pixel is determined according to the brightness calculation value.
3. The image processing method according to claim 2, characterized in that: The preset value includes a first preset value and a second preset value, and the first preset value is different from the second preset value; The method of determining the brightness calculation value by taking the Y component of the pixel as a base and a preset value as an exponent comprises: Taking the Y component of the pixel as a base and taking the first preset value and the second preset value as exponents respectively, determining a first brightness calculation value and a second brightness calculation value; The determining, according to the brightness calculation value, a second brightness component corresponding to the pixel includes: According to a preset suppression coefficient, the first brightness calculation value and the second brightness calculation value are linearly fused to obtain a second brightness component corresponding to the pixel.
4. The image processing method according to claim 1, characterized in that: The performing enhancement processing on the first brightness component of the pixel to obtain a third brightness component corresponding to the pixel comprises: Performing curve stretching on the first brightness component of the pixel to obtain a stretched brightness component corresponding to the pixel; According to a preset stretching coefficient, the first brightness component corresponding to the pixel and the stretched brightness component are linearly fused to obtain a third brightness component corresponding to the pixel.
5. The image processing method according to any one of claims 1 to 4, characterized in that: The step of acquiring a facial region image comprises: Acquire an image to be processed, perform facial region detection on the image to be processed, and obtain an initial facial region image; Reducing the initial facial region image based on a preset ratio to obtain a first facial region image; Performing Gaussian blur processing on the first facial region image to obtain a second facial region image; The second facial region image is scaled back to an original size to obtain the facial region image; the original size is the size of the initial facial region image.
6. The image processing method according to any one of claims 1 to 4, characterized in that: The method further comprises: According to the key point information in the target facial region image, the preset makeup material is superimposed on the target facial region image.
7. An image processing device, characterized in that: include: A first brightness component determination unit is configured to acquire a facial region image and determine a first brightness component of pixels in the facial region image in a preset color space; a brightness suppression unit, configured to suppress the brightness of the first brightness component of the pixel to obtain a second brightness component corresponding to the pixel; The second brightness component is smaller than the first brightness component, and the degree of brightness suppression is negatively correlated with the first brightness component; a brightness enhancement unit, configured to perform enhancement processing on the first brightness component of the pixel to obtain a third brightness component corresponding to the pixel; a brightness fusion unit, configured to determine a first product of a first fusion coefficient and a first brightness component corresponding to the pixel, and a second product of a second fusion coefficient and a third brightness component corresponding to the pixel; the second fusion coefficient is the second brightness component corresponding to the pixel, and the sum of the first fusion coefficient and the second fusion coefficient is 1; determine the sum of the first product and the second product to obtain a fused brightness component corresponding to the pixel; The replacement unit is configured to replace the first brightness component of the pixel in the facial area image with the fused brightness component corresponding to the pixel to obtain a target facial area image.
8. The image processing device according to claim 7, characterized in that: The preset color space is the YCbCr color space; The brightness suppression unit comprises: A brightness calculation unit is configured to determine a brightness calculation value using the Y component of the pixel as a base and a preset value as an exponent; the preset value is greater than 1; The second brightness component determining unit is configured to determine a second brightness component corresponding to the pixel according to the brightness calculation value.
9. The image processing device according to claim 8, characterized in that: The preset value includes a first preset value and a second preset value, and the first preset value is different from the second preset value; The brightness calculation unit is specifically configured to determine a first brightness calculation value and a second brightness calculation value by taking the Y component of the pixel as a base and taking the first preset value and the second preset value as exponents respectively; The second brightness component determination unit is specifically configured to perform linear fusion of the first brightness calculation value and the second brightness calculation value according to a preset suppression coefficient to obtain a second brightness component corresponding to the pixel.
10. The image processing device according to claim 7, characterized in that: The brightness enhancement unit comprises: a stretching unit, configured to perform curve stretching on the first brightness component of the pixel to obtain a stretched brightness component corresponding to the pixel; The linear fusion unit is configured to perform linear fusion on the first brightness component corresponding to the pixel and the stretched brightness component according to a preset stretching coefficient to obtain a third brightness component corresponding to the pixel.
11. The image processing device according to any one of claims 7 to 10, characterized in that: The first brightness component determining unit comprises: A facial region detection unit is configured to acquire an image to be processed, perform facial region detection on the image to be processed, and obtain an initial facial region image; A first scaling unit is configured to perform a reduction process on the initial facial region image based on a preset ratio to obtain a first facial region image; A Gaussian blur processing unit is configured to perform Gaussian blur processing on the first facial region image to obtain a second facial region image; The second scaling unit is configured to scale the second facial region image back to an original size to obtain a facial region image; the original size is the size of the initial facial region image.
12. The image processing device according to any one of claims 7 to 10, characterized in that: The device also includes: The material superimposing unit is configured to superimpose the preset makeup material onto the target facial area image according to the key point information in the target facial area image.
13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing method according to any one of claims 1 to 6. 14 . A computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method according to claim 1 .
15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Human visual perception simulation-based self-adaptive low-illumination image enhancement method
CN105046663A
Image fusion processing method and device
CN110298812A
