Portrait enhancement model training method and related device

Abstract

The application discloses a portrait enhancement model training method and related equipment, the method comprises the following steps: obtaining a label image, a sample image and a first prior value, the label image and the sample image are images processed by an image signal processor (ISP), the noise of the sample image is greater than that of the label image, and the first prior value is used to represent the noise size of the sample image; model training is performed based on the label image, the sample image and the first prior value, and a portrait enhancement model is obtained, the portrait enhancement model is used to generate facial texture of a face in an image, and the size of the first prior value is inversely proportional to the amount of generated facial texture. By using the method, the facial texture can be more accurately generated, and the image quality is improved.

Description

Technical Field

[0001] This application relates to the field of computer science, and in particular to methods and equipment for training human image enhancement models. Background Technology

[0002] Currently, mobile phone selfies often undergo portrait enhancement to produce clearer images. This enhancement typically involves generating textures for the face. However, current methods suffer from issues such as localized smoothing of facial textures, poor texture detail, and severe blurring, leading to image degradation. Summary of the Invention

[0003] This application provides a method and related equipment for training a human face enhancement model, which can generate facial textures more accurately and improve image quality.

[0004] Firstly, some embodiments of this application provide a method for training a portrait enhancement model. This method may include: acquiring a label image, a sample image, and a first prior value, wherein the label image and the sample image are images processed by an image processor (ISP), the noise level of the sample image is greater than that of the label image, and the first prior value is used to characterize the noise level of the sample image; training a model based on the label image, the sample image, and the first prior value to obtain a portrait enhancement model, wherein the portrait enhancement model is used to generate facial textures for faces in an image, and the magnitude of the first prior value is inversely proportional to the amount of facial texture generated.

[0005] By using images processed by ISP as training data, the trained portrait enhancement model can be more closely adapted to electronic devices (e.g., mobile phones) that use ISP for image processing. Furthermore, a first prior value is introduced during training. Since this first prior value represents the noise level of the image, a larger value indicates less noise. This first prior value is inversely proportional to the amount of facial texture. Therefore, introducing the first prior value can supervise the model to generate more facial texture when the image noise is high, thereby achieving more accurate facial texture generation and improving image quality.

[0006] In one possible implementation, obtaining the label image and the sample image includes: obtaining an initial image, which is a noise-free image; performing inverse ISP processing and ISP processing on the initial image sequentially to obtain the label image; and performing inverse ISP processing, noise addition processing, and ISP processing on the initial image sequentially to obtain the sample image.

[0007] By using the above method, we can obtain label images and sample images that have both been processed by ISP. The sample images are obtained after adding noise, so the noise in the sample images is greater than that in the label images.

[0008] In one possible implementation, acquiring the label image and the sample image includes: taking a picture based on a first sensitivity, performing ISP processing on the captured image to obtain the label image; taking a picture based on a second sensitivity, performing ISP processing on the captured image to obtain the sample image; wherein the first sensitivity is less than the second sensitivity.

[0009] The above method shows that images captured at different ISO sensitivities have different levels of noise. The higher the ISO sensitivity, the greater the noise in the captured image. Therefore, the image captured at a higher ISO sensitivity is used as the sample image, and the second image captured at a lower ISO sensitivity is used as the label image, so that the noise in the label image is less than that in the sample image.

[0010] In one possible implementation, a first image and a second prior value are obtained. The first image is an image captured by the camera after ISP processing, and the second prior value is used to characterize the noise level of the first image. The first image and the second prior value are input into the portrait enhancement model to obtain a second image, in which the face has more facial texture than the face in the first image.

[0011] By incorporating prior values ​​into the training process, these prior values ​​can supervise the image enhancement model to adjust for noise levels in the first image and generate different facial textures during the application phase of the image enhancement model.

[0012] In one possible implementation, the second prior value is a matrix, and the values ​​in the matrix correspond one-to-one with the regions in the first image.

[0013] Using the above method, when the first prior value is a matrix, different degrees of facial texture generation can be performed on each region of the image. For example, more textures can be generated for noisy regions in the same image, while fewer textures can be generated for less noisy regions in the same image.

[0014] In one possible implementation, the second prior value is determined based on the shooting mode corresponding to the first image and the first image itself.

[0015] In this way, the determination of the second prior value takes into account the shooting mode, thus allowing different second prior values ​​to be determined for different shooting modes. Furthermore, the second prior value is inversely proportional to the generated facial texture, enabling the generation of different facial textures for different shooting models. For example, in beauty mode, fewer facial textures can be generated to achieve a skin-smoothing effect.

[0016] In one possible implementation, the first image is subjected to face segmentation processing to obtain a face segmentation result, which includes annotations of each region of the face; the first image and a second prior value are obtained, including: determining the second prior value based on the face segmentation result, the shooting mode, and the first image.

[0017] Using the methods described above, different levels of facial texture can be generated for different areas of the face.

[0018] Secondly, this application provides a human image enhancement model training device. This device can be an electronic device, a component within an electronic device, or a device compatible with an electronic device. The human image enhancement model training device can also be a chip system, capable of executing the methods performed by the electronic device in the first aspect. The functions of the human image enhancement model training device can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the aforementioned functions. These units can be software and / or hardware. The operations and beneficial effects performed by the human image enhancement model training device are described in the first aspect and their beneficial effects are not repeated here.

[0019] Thirdly, this application provides an electronic device including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, including computer instructions, which, when executed by the one or more processors, cause the electronic device to perform the human image enhancement model training method in any possible implementation of the first aspect described above.

[0020] Fourthly, this application provides a chip system including a processor and an interface, the processor and the interface being coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions to perform the human image enhancement model training method in any possible implementation of the first aspect above.

[0021] Fifthly, this application provides a computer-readable storage medium storing a computer program / instructions that, when the computer program product is run on a computer, cause the computer to execute the human image enhancement model training method in any possible implementation of the first aspect described above.

[0022] Sixthly, this application provides a computer program product that, when run on a computer, causes the computer to execute the human image enhancement model training method in any possible implementation of the first aspect above. Attached Figure Description

[0023] Figure 1 A flowchart illustrating a method for training a human image enhancement model provided in an embodiment of this application;

[0024] Figure 2 A schematic diagram illustrating the process of acquiring sample images and label images provided in an embodiment of this application;

[0025] Figure 3A A schematic diagram of a matrix and image region provided in an embodiment of this application;

[0026] Figure 3B A schematic diagram of a beauty mode interface provided in an embodiment of this application;

[0027] Figure 3C A schematic diagram of an image region provided in an embodiment of this application;

[0028] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0029] Figure 5 A schematic diagram of the software structure of an electronic device provided in an embodiment of this application;

[0030] Figure 6 This application provides a schematic diagram of the application process for a human face enhancement model.

[0031] Figure 7 This is a schematic diagram of the structure of a human image enhancement model device provided in an embodiment of this application;

[0032] Figure 8 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0034] It should be understood that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0035] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0036] I. Image Signal Processor (ISP)

[0037] The Image Signal Processor (ISP) plays a crucial role in digital imaging systems. Located between the image sensor and the output device, it implements a series of image processing algorithms to transform the raw electrical signals captured by the sensor into a visualized image. An ISP can include one or more of the following processing methods: Black Level Compensation, Lens Shading Correction (LSC), White Balance (WB), Bad Pixel Correction, Demosaurus, Bayer noise reduction, White Balance (AWB) correction, Color Correction, and Gamma Correction. The ISP can also include more processing methods; there are no limitations on this. Typically, RAW images are processed by the ISP to obtain RGB images.

[0038] RAW images refer to images in RAW format. RAW format is an uncompressed image file format that retains all the data captured by the camera sensor. The advantage of RAW format is that it can record the details of the photo and retain more image information, including exposure, tone, and style.

[0039] 2. Sensitivity (International Organization for Standardization, ISO)

[0040] ISO sensitivity is an indicator used in photography to measure the sensitivity of a camera's image sensor to light, usually expressed as an ISO value. A higher ISO value indicates greater sensitivity, allowing the camera to capture more light in darker environments. ISO sensitivity, along with aperture and shutter speed, determines the overall exposure of a photograph.

[0041] Exposure refers to the total amount of light received by a camera's image sensor. To maintain a fixed exposure, increasing the ISO value allows you to decrease the aperture or shorten the exposure time to achieve the same result.

[0042] At the same exposure level, the higher the ISO value, the greater the noise in the captured image; conversely, at the same exposure level, the lower the ISO value, the less noise in the captured image.

[0043] The following is combined Figure 1 This application provides a detailed description of a method for training a human image enhancement model, as illustrated in the embodiments of this application. Figure 1 As shown, the training method for the human image enhancement model is as follows: steps 101 and 102. Figure 1 The method shown can be executed by an electronic device. Or Figure 1 The subject of the method shown can be a chip or chip system in an electronic device, but this application does not limit it. Figure 1 The method will be explained using an electronic device as the executing entity. Specifically:

[0044] 101. The electronic device acquires a label image, a sample image, and a first prior value. The label image and the sample image are images processed by an image processor (ISP). The noise of the sample image is greater than that of the label image. The first prior value is used to characterize the noise level of the sample image.

[0045] Optionally, both the sample image and the label image may include a human face.

[0046] Optionally, the noise in the sample image is greater than that in the label image, or in other words, the facial texture in the sample image is less than that in the label image.

[0047] Optionally, both the label image and the sample image can be in sRGB format. They can be images with different noise levels but identical visual quality. These label and sample images are used for model training.

[0048] The label image and sample image can be obtained through, but are not limited to, the following two methods, which are described below:

[0049] Method 1: Obtain by using a degenerate method.

[0050] In one possible embodiment, the electronic device acquires an initial image, which is a noise-free image; the electronic device sequentially performs anti-ISP processing and ISP processing on the initial image to obtain a label image; the electronic device sequentially performs anti-ISP processing, noise addition processing, and ISP processing on the initial image to obtain a sample image.

[0051] Optionally, the initial image is an sRGB image of a portrait captured by a DSLR camera, and this initial image is a clean and noise-free image.

[0052] Optionally, the initial image can be an image captured by an electronic device, an image captured by a DSLR camera and then sent to the electronic device, or an image downloaded from the Internet.

[0053] Optionally, the electronic device acquires an initial image; the electronic device performs inverse ISP processing on the initial image to obtain a degraded image; the electronic device performs ISP processing on the degraded image to obtain a label image; the electronic device performs noise addition processing and ISP processing on the degraded image to obtain a sample image.

[0054] Optionally, the electronic device acquires an initial image; the electronic device uses the initial image as a label image; the electronic device sequentially performs anti-ISP processing, noise addition processing, and ISP processing on the initial image to obtain a sample image.

[0055] Optionally, other processing may be included between anti-ISP processing and ISP processing. Similarly, other processing may also be included between anti-ISP processing, noise addition processing, and ISP processing. This application does not impose any restrictions on this.

[0056] Optionally, ISP processing may include multiple modules (Gamma module, LSC module, etc.), and the number and type of modules included in anti-ISP processing are the same as those included in ISP processing. For example, if ISP processing includes an LSC module, anti-ISP processing will include an iLSC module. For example, if ISP processing includes six modules, then anti-ISP processing also includes six modules, and the six modules included in ISP processing and anti-ISP processing are the same.

[0057] For example, such as Figure 2 As shown, Figure 2 The sRGB in the image is the initial image. The process of obtaining the label image and sample image will perform anti-ISP processing (iTM module, iGamma module, iCCM module, iLSC module, iWB module and Moaisc module) and ISP processing (TM module, Gamma module, CCM module, LSC module, WB module and DeMoaisc module) on the initial image.

[0058] In this module, ISP processing and anti-ISP processing are paired one-to-one. For example, the TM module corresponds to the iTM module. The TM module is mainly used for color correction to optimize color performance and detail clarity in the image. The iTM module restores the colors of the original scene using an algorithm that is the opposite of that of the TM module.

[0059] The Gamma module corresponds to the iGamma module. The Gamma module is used for gamma correction to adjust the contrast and brightness of the image, making the image more consistent with the visual characteristics of the human eye. The iGamma module is mainly used for inverse gamma correction to restore the most original linear brightness information in the image data.

[0060] The CCM module corresponds to the iCCM module. The CCM module is primarily used for color correction to adjust and optimize the color representation of images. Because the sensor's response to the spectrum differs from the human eye's, CCM is needed to correct these differences, making the captured image colors closer to what the human eye sees. The iCCM module is mainly used to restore the image to a state close to the sensor's original output.

[0061] The LSC module corresponds to the iLSC module. The LSC module primarily addresses the issue of uneven image brightness and color caused by lens optical characteristics. For example, due to lens optical properties, the light intensity received by the edge areas of the sensor's image area is typically weaker than that of the center area, resulting in a bright center and dark edges in the image. LSC compensates for image edges through software, thus ensuring that the brightness and color of the edges are consistent with the center area. The iLSC module restores the image brightness and color to the original output state of the sensor.

[0062] The WB module corresponds to the iWB module. The WB module adjusts the color balance of an image to ensure that the colors in the image reflect the original scene as realistically as possible under different light sources. Since different light sources (such as sunlight, fluorescent lamps, and incandescent lamps) have different color temperatures, this can cause color deviations in the captured images. For example, a photo taken under fluorescent light might appear greenish, while a photo taken under incandescent light might appear yellowish. To correct these color deviations, the WB module adjusts the gain of the red, green, and blue channels in the image according to the color temperature of the light source, thus making the image colors more natural and accurate. The iWB module is used to adjust the image colors back to the original output state of the sensor.

[0063] The Moaisc module corresponds to the DeMoaisc module. The Moaisc module is used to suppress noise in an image. It uses algorithms to identify and reduce random noise in the image, including electronic interference from the image sensor during image capture. The DeMoaisc module is used to recover the noise from the image.

[0064] Optionally, the sample image acquisition process includes an additional noise-adding module compared to the label image acquisition process. Therefore, the ISP processing in the sample image acquisition process includes a noise reduction module. That is... Figure 2 The denoising module is part of the ISP processing. For example, the ISP processing in the acquisition process of this sample image includes the following modules: denoising module, Moaisc module, TM module, Gamma module, CCM module, LSC module, and WB module.

[0065] Optionally, the ISP processing of the label image may also include a denoising module. For example, the ISP processing in the label image acquisition process may include the following modules: denoising module, Moaisc module, TM module, Gamma module, CCM module, LSC module, and WB module.

[0066] Optionally, in addition to the modules mentioned above, ISP processing and anti-ISP processing may include more or fewer modules, or some of the modules in ISP processing and anti-ISP processing may be other modules.

[0067] It's important to note that this ISP processing is not linear. This means that the image after the reverse ISP processing and subsequent ISP processing will be different from the image before processing. In other words, the label image and the initial image are different (specifically, they may differ in color, etc.). To ensure alignment between the label image and the sample image, the label image needs to undergo the same reverse ISP and ISP processing as the sample image.

[0068] In one possible embodiment, the noise-adding module adds noise by calibrating the noise of a camera, specifically the camera of the electronic device to which the portrait enhancement model is applied. That is, the noise-adding module can simulate the noise carried in images captured by the camera of the electronic device to which the portrait enhancement model is applied. For example, if the portrait enhancement model is applied to a mobile phone, the noise in the image obtained after the noise-adding module adds noise is similar to the noise in images captured by the mobile phone's camera.

[0069] In one possible embodiment, the noise added to the image by this noise-adding model can be approximately represented as Poisson + Gaussian, and the noise-adding image satisfies the following formula:

[0070]

[0071] Among them, I n This refers to the image after noise addition. For the purposes of this application, this I... n Specifically, it can be the image obtained by sequentially performing inverse ISP processing and noise addition processing on the initial image (e.g.) Figure 2 (Image output by the noise-adding module), I cThis refers to the image before noise addition; for the purposes of this application, this I... c Specifically, it can be the image after inverse ISP processing of the initial image (e.g.) Figure 2 (Image output from Mosaic). σ is the standard deviation of Gaussian noise, and K is the gain value in the Poisson distribution.

[0072] Method 2: Obtain the image using different photosensitive methods.

[0073] In one possible embodiment, the electronic device acquires a tag image and a sample image by: taking a picture based on a first sensitivity and then performing ISP processing on the captured image to obtain a tag image; taking a picture based on a second sensitivity and then performing ISP processing on the captured image to obtain a sample image; wherein the first sensitivity is less than the second sensitivity.

[0074] In this case, the exposure levels for the two shots can be the same. Under the same exposure level, the sensitivity during shooting is proportional to the noise of the captured image. That is, the first sensitivity is less than the second sensitivity. Therefore, the noise of the label image captured based on the first sensitivity is less than the noise of the sample image captured based on the second sensitivity.

[0075] Optionally, since the exposure levels for both shots are the same, and the exposure level is determined by both ISO and exposure time, the first ISO is lower than the second ISO, and the first exposure time is longer than the second exposure time. An image captured with a long exposure (long exposure time) and a low ISO can be considered a noise-free image.

[0076] Optionally, the second sensitivity can be any value between 1600 and 19200. The first sensitivity can be 100.

[0077] Optionally, the image captured based on the first sensitivity and the image captured based on the second sensitivity are two images of the same scene. That is, although the images are obtained from two separate captures, they need to be from the same scene in order to ensure that the resulting label image and sample image are as similar as possible.

[0078] Optionally, a tripod mode can be used to shoot based on the first ISO and the second ISO. This tripod mode can be roughly understood as the camera not shaking when shooting, avoiding the difference between the two photos caused by hand-held shooting.

[0079] Optionally, the shooting mode can be a selfie mode.

[0080] 102. The electronic device trains a model based on the labeled image, sample image and first prior value to obtain a portrait enhancement model. The portrait enhancement model is used to generate facial textures for faces in the image. The magnitude of the first prior value is inversely proportional to the amount of facial textures generated.

[0081] The first prior value characterizes the noise level in the image. A larger first prior value indicates less noise in the sample image, and vice versa. The magnitude of this first prior value is inversely proportional to the amount of facial texture generated. Therefore, this first prior value controls the model to generate more facial textures when the image noise is high, ensuring a direct proportionality between noise and facial texture. Thus, by introducing this first prior value during model training, the model learns the relationship between noise and generated facial textures. Consequently, when applying this portrait enhancement model, it can generate more facial textures for noisy images and less for less noisy images.

[0082] Optionally, this first prior value is used to control the intensity of the texture generated by the model.

[0083] Optionally, the first prior value can be the value output by the denoising module, for example... Figure 2 As shown, the first prior value is output to the model by the denoising module for model training.

[0084] Optionally, the calculation method of the first prior value can be preset in the denoising module, that is, the algorithm of the first prior value is built into the denoising module.

[0085] In one possible embodiment, the first prior value satisfies the following formula:

[0086]

[0087] Among them, P t I represents the first prior value. n This refers to the image after noise addition. For the purposes of this application, this I... n Specifically, it can be the image obtained by sequentially performing inverse ISP processing and noise addition processing on the initial image (e.g.) Figure 2 (Image output by the noise-adding module), I c This refers to the image before noise addition; for the purposes of this application, this I... c Specifically, it can be the image after inverse ISP processing of the initial image (e.g.) Figure 2 (Image output from Mosaic). σ is the standard deviation of Gaussian noise, and K is the gain value in the Poisson distribution. This represents the noisy image corresponding to the minimum gain. As can be seen from Formula 2, the greater the noise in the image, the lower the gain.n The larger the value, the smaller the first prior value. The first prior value is inversely proportional to the noise in the image.

[0088] In one possible embodiment, the electronic device trains a model based on a label image, a sample image, and a first prior value. Specifically, the sample image is input into the model to obtain an enhanced sample image output by the model, the enhancement process being supervised by the first prior value; the loss value between the enhanced sample image and the label image is calculated, and the parameters in the model are changed based on the loss value.

[0089] The smaller the loss value, the better the trained model. When the loss value is less than a certain preset threshold, the model is considered to have been trained and a human portrait enhancement model is obtained.

[0090] In one possible embodiment, the first prior value is a matrix, and the values ​​in the matrix correspond one-to-one with the regions in the sample image.

[0091] Optionally, the magnitude of the first value is inversely proportional to the amount of facial texture generated in the first region. This first value is any value in the first prior value matrix, which characterizes the noise level in the first region.

[0092] Optionally, the regions in the sample image can be evenly divided. The method of dividing the sample image regions can be pre-set. For example, if the sample image is divided into 8*8 regions, which is 64 regions, then the matrix is ​​also in 8*8 form. This matrix also includes 64 values, each value corresponding to one region, that is, 64 values ​​and 64 regions correspond one-to-one. Each value represents the noise corresponding to that region, and the magnitude of each value is inversely proportional to the amount of facial texture generated in that region.

[0093] The above combination Figure 1 The training process of the model has been introduced. The application of the resulting portrait enhancement model will be further described below. It should be noted that the training and application of this portrait enhancement model can be performed on the same device or on different devices. For example, the training phase of the portrait enhancement model may be conducted on a computer before the phone leaves the factory; while the application phase may be implemented on the phone after it leaves the factory. In other words, the portrait enhancement module is trained before leaving the factory and then built into the phone, allowing users to use the trained portrait enhancement model through their phones.

[0094] The following is a further introduction to the application of human image enhancement models:

[0095] In one possible embodiment, the electronic device acquires a first image and a second prior value. The first image is an image captured by the camera after ISP processing, and the second prior value is used to characterize the noise level of the first image. The first image and the second prior value are input into a portrait enhancement model to obtain a second image, in which the face has more facial texture than the face in the first image.

[0096] The second prior value characterizes the noise level in the first image. A larger second prior value indicates less noise in the first image, and vice versa. The magnitude of the second prior value is inversely proportional to the amount of facial texture generated. Therefore, a larger second prior value results in fewer facial textures generated in the second image, and vice versa.

[0097] Optionally, the modules included in the ISP processing during the application phase are the same as those included in the ISP processing during the training phase. For example, the ISP processing during the training phase includes the following modules: denoising module, Moaisc module, TM module, Gamma module, CCM module, LSC module, and WB module. The ISP processing during the application phase also includes the following modules: denoising module, Moaisc module, TM module, Gamma module, CCM module, LSC module, and WB module.

[0098] Optionally, the additional facial textures in the second image compared to the first image are generated by a portrait enhancement model. In other words, the increased facial textures in the second image are generated by this portrait enhancement model.

[0099] Optionally, the noise in the second image is less than that in the first image, or in other words, the second image is clearer than the first image.

[0100] In one possible embodiment, the second prior value is a matrix, and the values ​​in the matrix correspond one-to-one with the regions in the first image.

[0101] Optionally, the magnitude of the second value is inversely proportional to the amount of facial texture generated in the second region. This second value is any value in the second prior value matrix, which characterizes the noise level in the second region.

[0102] Optionally, the region in the first image can be evenly divided, and the method of dividing the first image region can be preset.

[0103] Optionally, the first image can be divided in the same way as the training sample images. For example, if the training sample images are divided into 8*8 equal parts, then the first image can also be divided into 8*8 equal parts. Similarly, the form of the second prior value is the same as that of the first prior value. For example, if the first prior value is an 8*8 matrix, then the second prior value is also an 8*8 matrix.

[0104] For example, such as Figure 3A As shown, Figure 3A The matrix marked with 301 is the second prior value. Figure 3A The image marked with 302 is the first image. (By...) Figure 3A As can be seen, any value in matrix 301 can be found in the corresponding region of the first image 302. The value "23" circled in matrix 301 corresponds to the darkened region in the first image.

[0105] In one possible embodiment, the second prior value is determined based on the shooting mode corresponding to the first image and the first image.

[0106] The shooting mode is the shooting mode used when capturing the first image. For example, the shooting mode can be: beauty mode.

[0107] Optionally, the calculation method for this second prior value can be found in Formula 2 above. Different shooting modes correspond to different values ​​of K and σ in Formula 2. The larger the values ​​of K and σ, the more facial textures are generated; conversely, the smaller the values ​​of K and σ, the fewer facial textures are generated.

[0108] Alternatively, the different shooting modes correspond to different values ​​of K in Formula 2. The larger the value of K, the more facial textures are generated, and vice versa.

[0109] Alternatively, the different shooting modes correspond to different values ​​of σ in Formula 2. The larger the value of σ, the more facial textures are generated, and vice versa.

[0110] In other words, the amount of texture generated can be controlled by controlling K and σ, or by controlling only K, or by controlling only σ.

[0111] Optionally, the electronic device determines the K and / or σ values ​​based on the shooting mode; and determines a second prior value based on the K and / or σ values ​​and the first image. Specifically, determining the K and / or σ values ​​based on the shooting mode can involve determining the K and / or σ values ​​corresponding to the current shooting mode based on a preset mapping relationship.

[0112] Optionally, the K and / or σ values ​​can be determined based on the parameters of the shooting mode. For example, when the shooting mode is beauty mode, the user can choose the size of the beauty filter. When the user chooses to turn the beauty filter to the maximum, the determined K and / or σ values ​​are the smallest, and the generated facial texture is the least. Conversely, when the user chooses to turn the beauty filter to the minimum, the determined K and / or σ values ​​are the largest, and the generated facial texture is the most.

[0113] For example, such as Figure 3B As shown, the user controls the amount of facial texture generated by sliding slider 303 to control the value of K and / or σ. When the user slides slider 303 to the right, the value of K and / or σ increases, and the second image has more facial texture; conversely, when the user slides slider 303 to the left, the value of K and / or σ decreases, and the second image has less facial texture.

[0114] Optionally, the second prior value is determined based on the mapping relationship between the shooting mode corresponding to the first image and the prior value of the shooting mode. Alternatively, it can be first determined whether the shooting mode is a special shooting mode (e.g., beauty mode). If the shooting mode is a special shooting mode, the second prior value is determined directly based on the mapping relationship between the shooting mode and the prior value of the shooting mode.

[0115] In one possible embodiment, the electronic device performs facial segmentation processing on the first image to obtain a facial segmentation result, the facial segmentation result including annotations of each region of the face; the electronic device acquires the first image and a second prior value, including: determining the second prior value based on the facial segmentation result, the shooting mode and the first image.

[0116] Optionally, the electronic device performs facial segmentation processing on the first image to obtain a facial segmentation result. Specifically, the electronic device obtains a face parsing result through a face parsing algorithm, which is the facial segmentation result.

[0117] Optionally, the labeling of each region of the face can be as follows: the region corresponding to the nose is labeled as "nose", the region corresponding to the eyes is labeled as "eyes", and so on.

[0118] Optionally, a label can correspond to one or more regions, for example Figure 3C As shown, the area labeled with an eye can correspond to... Figure 3C Two areas are deepened. The amount of facial texture generated can be controlled by adjusting the values ​​of K and / or σ.

[0119] Optionally, in one possible embodiment, the electronic device performs facial segmentation processing on the first image to obtain a facial segmentation result, the facial segmentation result including annotations of each region of the face; the electronic device acquires the first image and a second prior value, including: determining the second prior value based on the facial segmentation result and the first image.

[0120] The hardware structure of electronic devices is described below:

[0121] Please see Figure 4 , Figure 4 This is a schematic diagram of the hardware structure of the electronic device 100 provided in this application embodiment. The electronic device 100 can be an electronic device corresponding to the training phase or an electronic device corresponding to the application phase.

[0122] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0123] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0124] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0125] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0126] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system. The processor 110 retrieves the instructions or data stored in the memory, causing the electronic device 100 to execute the imaging method performed by the electronic device in the following method embodiments.

[0127] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0128] The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger.

[0129] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc. In some other embodiments, the power management module 141 may also be located in the processor 110.

[0130] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0131] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0132] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0133] A modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor.

[0134] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (BT), BLE broadcasting, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0135] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with networks and other devices through wireless communication technology.

[0136] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0137] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. In some embodiments, electronic device 100 may include one or N displays screens 194, where N is a positive integer greater than 1. Display screen 194 may include OLED screens.

[0138] Optionally, the display 194 may further include: an OLED glass layer, an OLED light-emitting unit, a fingerprint recognition sensor, a microlens array, etc. The display 194 supports optical in-display fingerprint recognition.

[0139] Electronic device 100 can perform shooting functions through an ISP, camera 193, video codec, GPU, display screen 194, and application processor. The ISP processes data fed back by the camera 193. The camera 193 captures still images or videos. The camera 193 may include a front-facing camera and a rear-facing camera; the front-facing camera is located on the display area of ​​the screen, and the rear-facing camera is located on the back area of ​​the screen. The digital signal processor processes digital signals, including digital image signals and other digital signals. The video codec is used to compress or decompress digital video. Electronic device 100 may support one or more video codecs.

[0140] NPU stands for Neural-Network (NN) Computing Processor. By drawing inspiration from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can quickly process input information and continuously learn on its own.

[0141] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to perform data storage functions.

[0142] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as a sound playback function), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data), etc. Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as flash memory devices.

[0143] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0144] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0145] A speaker 170A, also called a "loudspeaker," is used to convert audio electrical signals into sound signals. A receiver 170B, also called a "handpiece," is used to convert audio electrical signals into sound signals. A microphone 170C, also called a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. A headphone jack 170D is used to connect wired headphones. A pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be located on the display screen 194. A gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. A barometric pressure sensor 180C is used to measure barometric pressure. A magnetic sensor 180D includes a Hall effect sensor. An accelerometer 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). A distance sensor 180F is used to measure distance. A proximity sensor 180G may include, for example, a light-emitting diode (LED) and a photosensor. An ambient light sensor 180L is used to sense ambient light intensity. A fingerprint sensor 180H is used to collect fingerprints. Temperature sensor 180J is used to detect temperature. Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K detects touch operations applied to or near it. Bone conduction sensor 180M can acquire vibration signals. Buttons 190 include power button, volume buttons, etc. Motor 191 can generate vibration prompts. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card.

[0146] Furthermore, an operating system runs on top of the aforementioned components. Examples include iOS and Android. The operating system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100. It should be noted that although this application embodiment uses the Android system as an example for illustration, its basic principles are equally applicable to electronic devices with other operating systems.

[0147] The software structure of electronic device 100 is described below:

[0148] Figure 5This is a schematic diagram of the software structure of an electronic device 100 provided in an embodiment of this application. The software structure adopts a layered architecture, which divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In this embodiment, the operating system (taking Android as an example, with Android running on an AP) can be divided into four layers, from top to bottom: the application layer (APP), the application framework layer (FWK), the hardware abstraction layer (HAL), and the kernel layer.

[0149] The application layer can include a series of application packages. For example... Figure 5 As shown, the application package may include applications such as a camera and a gallery. In this embodiment, "camera" refers to a camera application. A camera application may include a camera interface module (which may be called a CameraApi2Module), etc. "Gallery" refers to a gallery application, which is used to store images and videos taken by electronic devices. This gallery application also provides users with playback functionality, allowing users to view historically captured images and videos within the gallery application.

[0150] The application framework layer provides application developers with an application programming interface (API) framework and various services and management tools to access core functionalities, including interface management, data access, application-layer messaging, application package management, telephony management, and location management. The application framework layer includes some predefined functions. For example... Figure 5 As shown, the application framework layer may include, but is not limited to, the camera service CameraService.

[0151] CameraService is responsible for scheduling the startup process of the camera application, creating and managing processes, and creating and managing windows. In this application, the portrait enhancement model can be built into the camera service or be a module independent of the camera service. For ease of description, we will take the example of the portrait enhancement model being built into the camera service.

[0152] The Hardware Abstraction Layer (HAL) is an interface layer located between the operating system kernel and the hardware circuitry. Its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform, providing the operating system with a virtual hardware platform. For example... Figure 5As shown, the hardware abstraction layer can include CameraResourceService, CameraProvider, etc. In addition, this hardware abstraction layer can also include: CameraDeviceSession, PreviewFlowImpl, etc. Specifically, CameraResourceService is used to interact with the memory modules in the hardware; CameraProvider is used to enumerate individual devices and manage their states, enabling the opening and closing of physical camera devices (such as rear cameras); CameraDeviceSession is used to create camera device sessions and store the attributes and configuration information required for the camera device sessions; PreviewFlowImpl is responsible for informing the app that the first frame of the preview has been displayed after the first frame of the preview is shown.

[0153] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, etc.

[0154] Based on the above Figure 4 and Figure 5 The following section further describes the application stages of the portrait enhancement model, as described in the previous section. The application stages of this portrait enhancement model mainly involve the following four parts: sensor (hardware), ISP (hardware), camera service (application framework layer), and image library. For example, we will take the portrait enhancement model built into the camera service. Figure 6 As shown:

[0155] First, after the sensor captures a raw image, it sends the acquired raw image to the ISP. The acquisition of the raw image by the sensor can be triggered by the user clicking the capture button or by a timed event; this application does not impose any restrictions on this.

[0156] Then, the ISP performs ISP processing on the raw image, which can be found above. Figure 2 ISP processing in, or more Figure 2 The ISP processing in the image may include more or fewer modules. After ISP processing of the raw image, a second image and a second prior value are obtained. The second prior value can be the output of the denoising module in the ISP, which calculates the prior value using the formula two mentioned above. The ISP processing then sends the first image and the second prior value to the camera service.

[0157] Then, the camera service performs portrait enhancement on the first image based on the second prior value to obtain the second image. Specifically, the camera service inputs the first image and the second prior value into the portrait enhancement model, which in turn inputs the second image.

[0158] Finally, the camera service sends the second image to the gallery, which then saves it.

[0159] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a human image enhancement model training device 700 provided in an embodiment of this application. Figure 7 The human image enhancement model training device shown can be an electronic device, a device within an electronic device, or a device that can be used in conjunction with an electronic device. Figure 7 The human image enhancement model training device shown may include an acquisition unit 701 and a processing unit 702. Wherein:

[0160] The acquisition unit 701 is used to acquire a label image, a sample image, and a first prior value. The label image and the sample image are images processed by the image processor ISP. The noise of the sample image is greater than that of the label image. The first prior value is used to characterize the noise level of the sample image.

[0161] The processing unit 702 is used to train the model based on the labeled image, the sample image and the first prior value to obtain the portrait enhancement model. The portrait enhancement model is used to generate facial textures for the face in the image. The magnitude of the first prior value is inversely proportional to the amount of facial texture generated.

[0162] In one possible implementation, the acquisition unit 701 is also used to acquire an initial image, which is a noise-free image;

[0163] The processing unit 702 is also used to sequentially perform anti-ISP processing and ISP processing on the initial image to obtain a label image; and to sequentially perform anti-ISP processing, noise addition processing and ISP processing on the initial image to obtain a sample image.

[0164] In one possible implementation, the acquisition unit 701 is further configured to perform ISP processing on the captured image after taking a picture based on a first sensitivity to obtain a label image; and to perform ISP processing on the captured image after taking a picture based on a second sensitivity to obtain a sample image; wherein the first sensitivity is less than the second sensitivity.

[0165] In one possible implementation, the acquisition unit 701 is further configured to acquire a first image and a second prior value, wherein the first image is an image after ISP processing of an image captured by the camera, and the second prior value is used to characterize the noise level of the first image;

[0166] The processing unit 702 is also used to input the first image and the second prior value into the portrait enhancement model to obtain a second image, wherein the face in the second image has more facial texture than the face in the first image.

[0167] In one possible implementation, the second prior value is a matrix, and the values ​​in the matrix correspond one-to-one with the regions in the first image.

[0168] In one possible implementation, the second prior value is determined based on the shooting mode corresponding to the first image and the first image itself.

[0169] In one possible implementation, the processing unit 702 is further configured to perform face segmentation processing on the first image to obtain a face segmentation result, the face segmentation result including annotations of each region of the face; and determine a second prior value based on the face segmentation result, the shooting mode and the first image.

[0170] For cases where the training device for the human image enhancement model can be a chip or a chip system, please refer to [link / reference]. Figure 8 The diagram shows the structure of the chip. Figure 8 The chip 800 shown includes a processor 801 and an interface 802. Optionally, it may also include a memory 803. The number of processors 801 can be one or more, and the number of interfaces 802 can be multiple.

[0171] For cases where the chip is used to implement the electronic device in the embodiments of this application:

[0172] The interface 802 is used to receive or output signals;

[0173] The processor 801 is used to perform data processing operations of the electronic device.

[0174] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0175] It is understood that some optional features in the embodiments of this application can be implemented independently in certain scenarios without relying on other features, such as the current solution on which they are based, to solve the corresponding technical problems and achieve the corresponding effects. Alternatively, they can be combined with other features as needed in certain scenarios. Accordingly, the audio data archiving device given in the embodiments of this application can also implement these features or functions, which will not be elaborated here.

[0176] It should be understood that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0177] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0178] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed on an electronic device, implement the functions of any of the above method embodiments.

[0179] This application also provides a computer program product that, when run on a computer, enables the computer to perform the functions of any of the above method embodiments.

[0180] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0181] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training a human image enhancement model, characterized in that, The method includes: Acquire a label image, a sample image, and a first prior value. The label image and the sample image are images processed by an image processor (ISP). The noise of the sample image is greater than that of the label image. The first prior value is used to characterize the noise level of the sample image and is inversely proportional to the noise level of the image. The model is trained based on the labeled image, the sample image and the first prior value to obtain a portrait enhancement model. The portrait enhancement model is used to generate facial textures for faces in the image. The magnitude of the first prior value is inversely proportional to the amount of facial textures generated. A first image and a second prior value are obtained. The first image is an image captured by the camera after ISP processing, and the second prior value is used to characterize the noise level of the first image. The first image and the second prior value are input into the portrait enhancement model to obtain a second image, in which the face has more facial texture than the face in the first image.

2. The method according to claim 1, characterized in that, The acquisition of the label image and sample image includes: Obtain an initial image, wherein the initial image is a noise-free image; The initial image is subjected to reverse ISP processing and the ISP processing sequentially to obtain a label image; The initial image is subjected to the anti-ISP processing, noise addition processing, and ISP processing in sequence to obtain a sample image.

3. The method according to claim 1, characterized in that, The acquisition of the label image and sample image includes: After taking a picture based on the first sensitivity, the captured image is processed by the ISP to obtain the label image; After taking a picture based on the second sensitivity, the captured image is processed by the ISP to obtain a sample image; The first photosensitivity is less than the second photosensitivity.

4. The method according to any one of claims 1-3, characterized in that, The second prior value is a matrix, and the values ​​in the matrix correspond one-to-one with the regions in the first image.

5. The method according to claim 4, characterized in that, The second prior value is determined based on the shooting mode corresponding to the first image and the first image.

6. The method according to claim 5, characterized in that, The method includes: The first image is subjected to facial segmentation processing to obtain facial segmentation results, which include annotations of each region of the face. The acquisition of the first image and the second prior value includes: Based on the face segmentation results, the shooting mode, and the first image, a second prior value is determined.

7. An electronic device comprising one or more memories and one or more processors, characterized in that, The memory is used to store a computer program; the processor is used to invoke the computer program, causing the electronic device to perform the method of any one of claims 1-6.

8. A chip system for use in electronic devices, characterized in that, The chip system includes at least one processor and an interface for receiving instructions and transmitting them to the at least one processor; the at least one processor executes the instructions to cause the electronic device to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1-6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1-6.

Images