Method of processing an image and apparatus therefor

Abstract

The application discloses a processing method and device of an image, and belongs to the technical field of communication. The method comprises the following steps: acquiring a first image and a near-infrared spectrum image, the near-infrared spectrum image and a face image region including a first object in the first image, converting the near-infrared spectrum image into a second image according to a parameter of the first image, and fusing the first image and the second image to obtain a third image, wherein the first image, the second image and the third image are all RGB images.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and specifically relates to an image processing method and apparatus. Background Technology

[0002] With the rapid development of electronic devices, people's focus on their functions has expanded beyond traditional communication to include other features such as entertainment and imaging. Especially with the rise of social media, people are using electronic devices more frequently for taking photos and are paying more attention to photo quality.

[0003] However, the sensors in feature-rich electronic devices are typically small. This smaller sensor size can limit light capture and the ability to render image details. This results in more image noise and loss of detail in portrait photos taken with electronic devices, thus highlighting imperfections. Consequently, portraits taken with electronic devices often contain more flaws. Summary of the Invention

[0004] The purpose of this application is to provide an image processing method and apparatus that can solve the technical problem of numerous defects in portraits captured by electronic devices.

[0005] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0006] Acquire a first image and a near-infrared spectral image, wherein the near-infrared spectral image and the first image include a facial image region of a first object;

[0007] The near-infrared spectral image is converted into a second image based on the parameters of the first image;

[0008] The first image and the second image are merged to obtain a third image, wherein the first image, the second image and the third image are all RGB images.

[0009] Secondly, embodiments of this application provide an image processing apparatus, the apparatus comprising:

[0010] The acquisition module is used to acquire a first image and a near-infrared spectral image, wherein the near-infrared spectral image and the first image include a face image region of a first object;

[0011] A conversion module is used to convert the near-infrared spectral image into a second image based on the parameters of the first image;

[0012] The fusion module is used to fuse the first image and the second image to obtain a third image, wherein the first image, the second image and the third image are all RGB images.

[0013] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method provided in the first aspect.

[0014] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method provided in the first aspect.

[0015] Fifthly, embodiments of this application provide a chip, which includes a processor and a communication interface, the communication interface and the processor being coupled together, the processor being used to run programs or instructions to implement the method provided in the first aspect.

[0016] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method as provided in the first aspect.

[0017] In the image processing method and apparatus of this application, a first image and a near-infrared spectral image can be acquired. The near-infrared spectral image and the first image contain a facial image region of a first object. The near-infrared spectral image is converted into a second image based on the color of the first image, and the first and second images are fused to obtain a third image. Since the near-infrared spectral image can eliminate small textures and blemishes in the facial image region, making the skin in the facial image region appear natural and smooth, the facial image region in the third image, obtained from the near-infrared spectral image and fused to obtain the third image, can also have small textures and blemishes eliminated, thereby beautifying the facial region in the image and reducing blemishes in the image. Attached Figure Description

[0018] Figure 1 This is a schematic flowchart of an image processing method provided in one embodiment of this application;

[0019] Figure 2 This is a schematic diagram of a product during the processing of an image processing method provided in one embodiment of this application;

[0020] Figure 3 This is a schematic diagram of a product during the processing of an image processing method provided in another embodiment of this application;

[0021] Figure 4This is a schematic diagram of a product during the processing of an image processing method provided in another embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in one embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application;

[0024] Figure 7 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0026] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0027] To address the aforementioned technical problems, this application provides an image processing method. The image processing method provided by this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0028] like Figure 1 As shown, Figure 1 This is a schematic flowchart illustrating an image processing method according to an embodiment of this application. This application provides an image processing method that may include:

[0029] S101, acquire a first image and a near-infrared spectral image, wherein the near-infrared spectral image and the first image include a face image region of the first object;

[0030] In this embodiment, an RGB image is a color image composed of three basic color channels: red, green, and blue. A first image including the face of the first object can be obtained by taking a picture of the face of the first object using a mobile phone, such as... Figure 2 As shown, the first image is a true color image, so the face image area will realistically display blemishes such as spots and wrinkles on the face.

[0031] Near-infrared spectral images are images acquired within the near-infrared spectral range. The near-infrared spectral range extends beyond the red end of the visible spectrum and is invisible to the human eye. The pixel values ​​of a near-infrared spectral image represent the reflectance or radiation characteristics of a person's face under near-infrared light. Near-infrared spectral images are obtained by capturing images of a person's face using a near-infrared camera.

[0032] S102, convert the near-infrared spectral image into a second image according to the parameters of the first image;

[0033] In this embodiment, since the near-infrared spectral image is obtained by capturing light in the near-infrared spectral range, and the first image is a color image composed of red, green and blue channels.

[0034] To fuse the first image and the infrared spectral image, the grayscale values ​​of the near-infrared spectral image can be mapped to the corresponding channels in the RGB color space. In this way, the near-infrared spectral image can present the appearance of a color image, thus obtaining the converted second image.

[0035] In the mapping process from a near-infrared spectral image to a second image, different mapping methods and algorithms can be used to adjust the relationship between grayscale values ​​and RGB channel values. These methods can be selected according to specific needs and objectives, such as linear mapping, logarithmic mapping, and histogram equalization.

[0036] S103, the first image and the second image are fused to obtain a third image, wherein the first image, the second image and the third image are all RGB images.

[0037] In this embodiment, the near-infrared spectral image captures energy within the near-infrared spectral range, which is invisible to the human eye. Compared to visible light, near-infrared light is scattered and absorbed to varying degrees when penetrating the skin, such as... Figure 3 As shown, in near-infrared spectral images, small textures and blemishes in the face image area can be eliminated, making the skin of the person in the face image area appear natural and smooth.

[0038] Since the second image is derived from a near-infrared spectral image, although it is an RGB image, it still retains the characteristics of a near-infrared spectral image. That is, compared to the first image obtained by directly using a color camera, the second image filters out smaller textures and blemishes in the face image area, making the skin appear more natural and smooth.

[0039] After obtaining the second image, Poisson editing can be used to merge the second image and the first image. Specifically, a first region can be identified in the first image, which can be an area in the face image region where there are blemishes or rough skin. Then, the content of the second image is seamlessly merged into the first region in the first image, making the skin in the first region more natural and smooth, and eliminating smaller textures and blemishes in the first region.

[0040] Specifically, the Poisson editing process can be represented by the following formula (1):

[0041] I1=f(I;N I ,M1), (1)

[0042] Where f(x) represents the Poisson editing formula, I1 represents the third image obtained by fusion, I represents the first image, and N I M1 represents the second image, and M1 represents the face image region.

[0043] In this application, a first image and a near-infrared spectral image can be acquired. The near-infrared spectral image and the first image contain a facial image region of a first object. The near-infrared spectral image is converted into a second image based on the color of the first image, and the first and second images are then fused to obtain a third image. Since the near-infrared spectral image can eliminate small textures and blemishes in the facial image region, making the skin in the facial image region appear natural and smooth, the facial image region in the third image, obtained from the near-infrared spectral image and fused to obtain the third image, can also have small textures and blemishes eliminated, thus beautifying the facial region. Compared to manual beautification, this automatic beautification method can quickly and efficiently beautify a large number of photos, improving beautification efficiency and saving time and costs.

[0044] In some embodiments, S102 includes:

[0045] The second image is obtained by spectral mapping of the near-infrared spectral image using the R, G, and B channels of the first image.

[0046] In this embodiment, since the first image is composed of three color channels—red (R), green (G), and blue (B)—the three color channels in the first image can be spectrally mapped to the near-infrared spectral image to obtain three mapping results. Thus, each of the three mapping results corresponds to information from one color channel in the first image, and these three mapping results represent an approximate representation of the near-infrared information in the near-infrared spectral image across the three color channels of the first image.

[0047] After obtaining the three mapping results, a new RGB image, namely the second image, can be obtained based on these three mapping results. The second image contains the result of mapping the near-infrared information of the near-infrared spectral image to the RGB channels, which is equivalent to fusing the near-infrared spectral image with the first image.

[0048] In this embodiment, the near-infrared spectral image can be mapped to the three channels of the RGB image, and the near-infrared information can be fused with the visible light information. This fusion process helps to extract and retain useful information in the near-infrared spectral image and fuse it into the first image, thereby obtaining a richer and more natural image representation.

[0049] In some embodiments, the near-infrared spectral image is spectrally mapped using the R, G, and B channels of the first image to obtain the second image, including:

[0050] N first sampling points are collected within a preset range of the first pixel in the near-infrared spectral image, wherein the first pixel is any pixel in the face region of the near-infrared spectral image, and N is a positive integer;

[0051] N second sampling points are collected within a preset range of the second pixel in the first single-channel image. The first single-channel image is any one of the images corresponding to the R, G, and B channels of the first image. The position of the second pixel in the first single-channel image is the same as the position of the first pixel in the near-infrared spectral image.

[0052] Determine the product of the N first sampling points and the N second sampling points, wherein the product includes P elements, where P is a positive integer;

[0053] Determine the first mean of the N first sampling points in the first color channel, the second mean of the N second sampling points in the near-infrared spectral channel, and the third mean of the P elements, wherein the first color channel is the color channel corresponding to the first single-channel image;

[0054] The near-infrared spectral image is spectrally mapped based on the first mean, the second mean, and the third mean to obtain a mapping result, wherein the mapping result is a set of single-channel pixels obtained by mapping based on the first single-channel image;

[0055] The second image is obtained by performing spectral mapping on the near-infrared spectral image based on the R, G, and B channels of the first image.

[0056] In this embodiment, the near-infrared spectral image is mapped onto three single-channel images, which actually involves mapping each pixel in the near-infrared spectral image. Specifically, during the mapping process, for the first pixel, N first sampling points can be randomly collected within a preset range of the first pixel, and N second sampling points can be randomly collected within a preset range of the second pixel.

[0057] The first average of the N first sampling points in the first color channel can characterize the feature information of the first pixel and the pixels near the first pixel. Similarly, the second average of the N second sampling points in the first color channel can characterize the feature information of the second pixel and the pixels near the second pixel.

[0058] By comparing the first and second means, the average difference between the feature information of the near-infrared spectral image and the single-channel image can be obtained. This difference represents the color shift between the near-infrared spectral image and the single-channel image. Furthermore, the average of the products of N first sampling points and N second sampling points, i.e., the third mean, can be calculated. The third mean reflects the correlation and contrast information between the two.

[0059] Therefore, after determining the first, second, and third means, the color shift between the near-infrared spectral image and the single-channel image, as well as the correlation and contrast between them, can be determined. Based on this information, the near-infrared spectral image can be mapped to obtain three mapping results, each of which is a set of pixels from one of the three channels: R, G, and B. Concatenating these three mapping results forms a new RGB image, the second image.

[0060] By mapping each pixel in the near-infrared spectral image in the above manner—that is, mapping based on the color shift, correlation, and contrast between the near-infrared spectral image and the single-channel image—the characteristics of the near-infrared spectral image can be better preserved, while fusing it into the single-channel image, thereby achieving a more accurate and natural mapping result.

[0061] In some embodiments, the near-infrared spectral image is mapped based on the first mean, the second mean, and the third mean to obtain a mapping result, including:

[0062] The first mapping coefficient and the second mapping coefficient are determined based on the first mean, the second mean, and the third mean;

[0063] A mapping equation is constructed based on the first mapping coefficient and the second mapping coefficient. The independent variable of the mapping equation is the second pixel point, and the dependent variable is the mapped pixel point. The mapped pixel point is the pixel point in the mapping result whose position corresponds to the second pixel point. The first mapping coefficient and the second mapping coefficient are parameters in the mapping equation.

[0064] In this embodiment, since each first pixel is mapped to a mapped pixel in the second image based on a second pixel, there is a mapping relationship between each second pixel and a mapped pixel. This mapping relationship can be represented by a mapping equation, which is shown below:

[0065] N R (p)=a·R(p)+b, (2)

[0066] Where p represents a pixel, N R (p) represents the mapped pixel, R(p) represents the second pixel, a is the first mapping coefficient, and b is the second mapping coefficient.

[0067] Therefore, by determining the first and second mapping coefficients, the mapping equation can be determined, and the mapping equation can be determined by the following formulas (3) and (4):

[0068]

[0069] b = μ R(p) -a·μ N(p） (4)

[0070] Where a is the first mapping coefficient, μ R(p) μ represents the second mean. N(p) μ represents the first mean. R⊙N(p) σ represents the third mean. 2 N(p) Then, represents the total variance of the N first sampling points in the near-infrared spectral image, and b is the second mapping coefficient.

[0071] Through the above calculations, both a and b can characterize the correlation between the first pixel and the second pixel.

[0072] In some embodiments, after S103, the method further includes:

[0073] At least one first defect point is obtained in the third image by the first detection model, and at least one second defect point is obtained in the near-infrared spectral image;

[0074] The third defect point in the third image is eliminated to obtain the fourth image, wherein the third defect point is the defect point in the third image that corresponds to the position of the second defect point.

[0075] In this embodiment, by fusing the first and second images to obtain the third image, some minor imperfections and textures are filtered out compared to the first image. The third image can be directly displayed on the device's screen as the final result of the beautification algorithm, or it can be further optimized.

[0076] Specifically, the first detection model can be used to continue detection in the third image to obtain at least one first blemish from the face image region in the third image. All detected first blemishes can be iterated over. If a second blemish exists in the near-infrared spectral image corresponding to the location of the first blemish in the third image, it indicates that these first blemishes are large blemishes that were not filtered out during image fusion. These first blemishes can then be identified as third blemishes, and the third blemishes in the third image can be eliminated to obtain a fourth image. The fourth image is a beautified image obtained through further optimization based on the third image.

[0077] The first detection model is a neural network model that uses the first blemish in a face image region as the object to be detected. It is trained on a large number of randomly generated simulation datasets, where the simulated data consists of face images. After training, the first detection model performs multi-layer convolution, pooling, and fully connected operations on the input images to extract image features. These features are then used for blemish detection and classification, and the detection result—the first blemish in each image—is output.

[0078] Furthermore, the process of obtaining the first defect point can be represented by the following formula (5):

[0079]

[0080] Where M2 is the defect area mask, and each defect area mask corresponds to a first defect point. This represents a small target detection model based on neural networks and confidence level, where I1 represents the third image and ∈ represents the set confidence level threshold.

[0081] When the confidence threshold is greater than the confidence threshold, the area can be considered a defective area mask, in which the first defect point exists.

[0082] In this embodiment, imperfections that cannot be filtered out by image fusion can be further filtered to obtain a more refined beautified portrait.

[0083] In some embodiments, eliminating the third defect in the third image to obtain the fourth image includes:

[0084] The defective areas in the third image are cropped, wherein the defective areas are the areas where the third defective point is located;

[0085] The fourth image is obtained by filling the cropped blemish area with the colors of the adjacent areas of the blemish area.

[0086] In this embodiment, after detecting the third blemish point in the third image, a blemish region of appropriate size can be determined centered on the third blemish point. Then, using Poisson editing technology, the color, texture, and structural information of the surrounding normal skin area are synthesized into the blemish region. By solving the Poisson equation, the pixel values ​​of the blemish region are filled to maintain the continuity and consistency of the synthesized result. Furthermore, if Poisson editing cannot completely repair the blemish region, other image inpainting algorithms can be used to further improve the result. Image inpainting algorithms can automatically fill the blemish region based on the content and statistical characteristics of the surrounding pixels, making it consistent with the surrounding image.

[0087] In some embodiments, after eliminating the third defect in the third image to obtain the fourth image, the method further includes:

[0088] The first image is subjected to bilateral filtering to obtain a smoothed image;

[0089] Facial texture is extracted from the first image based on the difference between the smoothed image and the first image;

[0090] The facial texture is added to the fourth image to obtain the fifth image.

[0091] In this embodiment, the fourth image, after removing the third blemish, can be displayed on the device's display interface as the final result of the beautification algorithm, or the fourth image can be further optimized.

[0092] Specifically, if a user feels that the smoothing effect of the fourth image obtained by the aforementioned beautification algorithm is too strong, and wants to make the face look more natural and realistic based on the fourth image, facial texture can be extracted from the facial texture in the face image region of the first image. The facial texture can be determined according to a pre-set rule, and the facial texture is added to the face image region of the fourth image to obtain the fifth image.

[0093] Specifically, before adding facial texture, the facial texture of the portrait needs to be obtained from the first image. Specifically, the first image can first be smoothed and noise reduced by bilateral filtering while maintaining edge sharpness, resulting in a smoothed image. Then, based on the difference between the first image and the smoothed image, the facial texture is extracted from the first image. Finally, the facial texture is added to the fourth image through weighted fusion to obtain the fifth image.

[0094] Specifically, the first image and the extracted facial texture are weighted and fused. The weighting controls the contribution of the facial texture to the final synthesized image. Typically, a weight image is used to determine the weight value of each pixel in the fusion process. After determining the weight value of each pixel, the first image and the facial texture are weighted and summed, and then fused according to the weight values ​​of the weight image. Pixels with higher weights retain more information from the facial texture, while pixels with lower weights retain more information from the first image.

[0095] For example, the process of extracting facial textures can be represented by the following texture extraction equation (6):

[0096] I detail =Ih(I;r h ), (6)

[0097] Among them, I detail For facial texture, h(I;r) h ) indicates that a radius of r is used in the first image. h The bilateral filtering is applied, where I represents the first image.

[0098] After obtaining the facial texture, the weighted fusion formula (7) can be expressed as:

[0099]

[0100] Among them, I detail For facial texture, I1 is the fifth image, and I2 is the fourth image.

[0101] Currently, beautification algorithms in related technologies are mainly divided into two categories: single-image beautification algorithms and multi-image beautification algorithms. Single-image beautification algorithms mostly smooth a single image through filtering operations, while multi-image beautification algorithms improve the beautification result by fusing images of different modalities.

[0102] Specifically, the smoothing intensity of single-image beautification algorithms is difficult to control. When the smoothing intensity is too low, blemishes are easily not removed cleanly, while when the smoothing intensity is too high, gradient inversion can easily occur at the edges. Existing multi-image beautification algorithms simply add the base layer of the RGB image to the detail layer of the NIR image to obtain the beautification result. This simple addition may lead to color distortion. Therefore, related technologies are prone to causing loss of detail or blurring of the beautified image.

[0103] Through the above methods, such as Figure 4 The fifth image shown combines the facial texture with the fourth image in a weighted manner, which can balance the facial texture with the original face image, so that the final synthesized image has both the detail and realism of the facial texture, and retains the shape and features of the original face.

[0104] Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in another embodiment of this application, as shown below. Figure 5 As shown, the image processing apparatus may include:

[0105] The acquisition module 501 is used to acquire a first image and a near-infrared spectral image, wherein the near-infrared spectral image and the first image include a face image region of a first object;

[0106] The conversion module 502 is used to convert the near-infrared spectral image into a second image according to the parameters of the first image;

[0107] The fusion module 503 is used to fuse the first image and the second image to obtain a third image, wherein the first image, the second image, and the third image are all RGB images. In this application, a first image and a near-infrared spectral image can be obtained. The near-infrared spectral image and the first image contain a facial image region of a first object. The near-infrared spectral image is converted into a second image based on the color of the first image, and the first image and the second image are fused to obtain the third image. Since the near-infrared spectral image can eliminate small textures and blemishes in the facial image region, making the skin in the facial image region appear natural and smooth, after obtaining the second image based on the near-infrared spectral image and fusing it to obtain the third image, the facial image region in the third image can also have small textures and blemishes eliminated, thereby beautifying the facial region in the image and reducing blemishes in the image.

[0108] In another alternative example, the conversion module 502 includes:

[0109] The mapping unit is used to perform spectral mapping on the near-infrared spectral image through the R, G, and B channels of the first image to obtain the second image.

[0110] In another alternative example, the mapping unit includes:

[0111] The first acquisition subunit is used to acquire N first sampling points within a preset range of the first pixel in the near-infrared spectral image, wherein the first pixel is any pixel in the face region of the near-infrared spectral image, and N is a positive integer;

[0112] The second acquisition subunit is used to acquire N second sampling points within a preset range of the second pixel in the first single-channel image, wherein the first single-channel image is any one of the images corresponding to the R, G, and B channels of the first image, and the position of the second pixel in the first single-channel image is the same as the position of the first pixel in the near-infrared spectral image.

[0113] A subunit is defined to determine the product of the N first sampling points and the N second sampling points, wherein the product includes P elements, where P is a positive integer;

[0114] A calculation subunit is used to determine the first mean of the N first sampling points in the first color channel, the second mean of the N second sampling points in the near-infrared spectral channel, and the third mean of the P elements, wherein the first color channel is the color channel corresponding to the first single-channel image;

[0115] A mapping subunit is configured to perform spectral mapping on the near-infrared spectral image based on the first mean, the second mean, and the third mean to obtain a mapping result, wherein the mapping result is a set of single-channel pixels obtained by mapping based on the first single-channel image;

[0116] A generation subunit is used to obtain the second image by performing spectral mapping on the near-infrared spectral image based on the R, G, and B channels of the first image.

[0117] In another alternative example, the image processing apparatus further includes:

[0118] The detection module is used to obtain at least one first defect in the third image through a first detection model, and to obtain at least one second defect in the near-infrared spectral image;

[0119] The elimination module is used to eliminate the third defect point in the third image to obtain a fourth image, wherein the third defect point is the defect point in the third image that corresponds to the position of the second defect point.

[0120] In another alternative example, the image processing apparatus further includes:

[0121] The filtering module is used to perform bilateral filtering on the first image to obtain a smoothed image;

[0122] An extraction module is used to extract facial texture from the first image based on the difference between the smoothed image and the first image;

[0123] An addition module is used to add the facial texture to the fourth image to obtain the fifth image.

[0124] The image processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific type of device.

[0125] The image processing device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0126] The image processing apparatus provided in this application embodiment can achieve Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0127] Optionally, such as Figure 6As shown in the illustration, this application also provides an electronic device 100, including a processor 110, a memory 119, and a program or instructions stored in the memory 119 and executable on the processor 110. The processor 110 includes a central processing unit (CPU) 1101 and a GPU compression processing unit 1102. When the program or instructions are executed by the processor 110, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, further details are omitted here.

[0128] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0129] Please refer to the following: Figure 7 , Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. The electronic device 100 includes, but is not limited to, components such as: a radio frequency unit 121, a network module 122, an audio output unit 123, an input unit 124, a sensor 125, a display unit 126, a user input unit 127, an interface unit 128, a memory 129, and a processor 120.

[0130] Those skilled in the art will understand that the electronic device 120 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 120 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0131] The input unit 124 is used to acquire a first image and a near-infrared spectral image, wherein the near-infrared spectral image and the first image include a face image region of a first object;

[0132] Processor 120 is configured to convert the near-infrared spectral image into a second image based on parameters of the first image;

[0133] The processor 120 is configured to fuse the first image and the second image to obtain a third image, wherein the first image, the second image, and the third image are all RGB images.

[0134] In this application, a first image and a near-infrared spectral image can be acquired. Both the near-infrared spectral image and the first image contain a facial image region of a first object. The near-infrared spectral image is converted into a second image based on the color of the first image. The first and second images are then fused to obtain a third image. Since the near-infrared spectral image can eliminate small textures and blemishes in the facial image region, making the skin in the facial image region appear natural and smooth, the facial image region in the third image, obtained from the near-infrared spectral image and fused with it, can also have small textures and blemishes eliminated, thereby beautifying the facial region in the image and reducing blemishes in the image.

[0135] In another alternative example, the processor 120 includes:

[0136] The mapping unit is used to perform spectral mapping on the near-infrared spectral image through the R, G, and B channels of the first image to obtain the second image.

[0137] In another alternative example, the mapping unit includes:

[0138] The first acquisition subunit is used to acquire N first sampling points within a preset range of the first pixel in the near-infrared spectral image, wherein the first pixel is any pixel in the face region of the near-infrared spectral image, and N is a positive integer;

[0139] The second acquisition subunit is used to acquire N second sampling points within a preset range of the second pixel in the first single-channel image, wherein the first single-channel image is any one of the images corresponding to the R, G, and B channels of the first image, and the position of the second pixel in the first single-channel image is the same as the position of the first pixel in the near-infrared spectral image.

[0140] A subunit is defined to determine the product of the N first sampling points and the N second sampling points, wherein the product includes P elements, where P is a positive integer;

[0141] A calculation subunit is used to determine the first mean of the N first sampling points in the first color channel, the second mean of the N second sampling points in the near-infrared spectral channel, and the third mean of the P elements, wherein the first color channel is the color channel corresponding to the first single-channel image;

[0142] A mapping subunit is configured to perform spectral mapping on the near-infrared spectral image based on the first mean, the second mean, and the third mean to obtain a mapping result, wherein the mapping result is a set of single-channel pixels obtained by mapping based on the first single-channel image;

[0143] A generation subunit is used to obtain the second image by performing spectral mapping on the near-infrared spectral image based on the R, G, and B channels of the first image.

[0144] In another alternative example, the image processing apparatus further includes:

[0145] The processor 120 is configured to acquire at least one first defect in the third image using a first detection model, and acquire at least one second defect in the near-infrared spectral image;

[0146] Processor 120 is configured to remove a third defect in the third image to obtain a fourth image, wherein the third defect is a defect in the third image that corresponds to the position of the second defect.

[0147] In another alternative example, the image processing apparatus further includes:

[0148] Processor 120 is configured to extract facial texture from the first image and add the facial texture to the fourth image to obtain a fifth image.

[0149] In another alternative example, the processor 120 is further configured to perform bilateral filtering on the first image to obtain a smoothed image;

[0150] Facial texture is extracted from the first image based on the difference between the smoothed image and the first image;

[0151] The facial texture is added to the fourth image to obtain the fifth image.

[0152] It should be understood that, in this embodiment, the input unit 124 may include a graphics processing unit (GPU) 1241 and a microphone 1242. The GPU 1241 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 126 may include a display panel 1261, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 127 includes at least one of a touch panel 1271 and other input devices 1272. The touch panel 1271 is also called a touch screen. The touch panel 1271 may include a touch detection device and a touch controller. Other input devices 1272 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0153] The memory 129 can be used to store software programs and various data. The memory 129 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 129 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 129 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0154] Processor 120 may include one or more processing units; optionally, processor 120 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 120.

[0155] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0156] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0157] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0158] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0159] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the image processing method embodiment described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0160] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0161] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0162] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method of processing an image, characterized by, include: Acquire a first image and a near-infrared spectral image, wherein the near-infrared spectral image and the first image include a facial image region of a first object; N first sampling points are collected within a preset range of the first pixel in the near-infrared spectral image, wherein the first pixel is any pixel in the face region of the near-infrared spectral image, and N is a positive integer; N second sampling points are collected within a preset range of the second pixel in the first single-channel image. The first single-channel image is any one of the images corresponding to the R, G, and B channels of the first image. The position of the second pixel in the first single-channel image is the same as the position of the first pixel in the near-infrared spectral image. Determine the product of the N first sampling points and the N second sampling points, wherein the product includes P elements, where P is a positive integer; Determine the first mean of the N first sampling points in the first color channel, the second mean of the N second sampling points in the near-infrared spectral channel, and the third mean of the P elements, wherein the first color channel is the color channel corresponding to the first single-channel image; The near-infrared spectral image is spectrally mapped based on the first mean, the second mean, and the third mean to obtain a mapping result, wherein the mapping result is a set of single-channel pixels obtained by mapping based on the first single-channel image; The second image is obtained by performing spectral mapping on the near-infrared spectral image based on the R, G, and B channels of the first image; The first image and the second image are fused to obtain a third image, wherein the first image, the second image and the third image are all RGB images.

2. The method of claim 1, wherein, After the step of fusing the first image and the second image to obtain the third image, the method further includes: At least one first defect point is obtained in the third image by the first detection model, and at least one second defect point is obtained in the near-infrared spectral image; The third defect in the third image is eliminated to obtain the fourth image, wherein the third defect is the defect in the first image that corresponds to the position of the second defect.

3. The method of claim 2, wherein, After eliminating the third defect in the third image to obtain the fourth image, the process further includes: The first image is subjected to bilateral filtering to obtain a smoothed image; Facial texture is extracted from the first image based on the difference between the smoothed image and the first image; The facial texture is added to the fourth image to obtain the fifth image.

4. An image processing apparatus characterized by comprising: include: The acquisition module is used to acquire a first image and a near-infrared spectral image, wherein the near-infrared spectral image and the first image include a face image region of a first object; The conversion module includes a first acquisition subunit, a second acquisition subunit, a determination subunit, a calculation subunit, a mapping subunit, and a generation subunit; The first acquisition subunit is used to acquire N first sampling points within a preset range of the first pixel in the near-infrared spectral image, wherein the first pixel is any pixel in the face region of the near-infrared spectral image, and N is a positive integer; The second acquisition subunit is used to acquire N second sampling points within a preset range of the second pixel in the first single-channel image, wherein the first single-channel image is any one of the images corresponding to the R, G, and B channels of the first image, and the position of the second pixel in the first single-channel image is the same as the position of the first pixel in the near-infrared spectral image. The determining subunit is used to determine the product of the N first sampling points and the N second sampling points, wherein the product includes P elements, where P is a positive integer; The calculation subunit is used to determine the first mean of the N first sampling points in the first color channel, the second mean of the N second sampling points in the near-infrared spectral channel, and the third mean of the P elements, wherein the first color channel is the color channel corresponding to the first single-channel image. The mapping subunit is used to perform spectral mapping on the near-infrared spectral image based on the first mean, the second mean, and the third mean to obtain a mapping result, wherein the mapping result is a set of single-channel pixels obtained by mapping based on the first single-channel image; The generation subunit is used to obtain a second image by performing spectral mapping on the near-infrared spectral image based on the R, G, and B channels of the first image; The fusion module is used to fuse the first image and the second image to obtain a third image, wherein the first image, the second image and the third image are all RGB images.

5. The apparatus of claim 4, wherein, The image processing apparatus further includes: The detection module is used to obtain at least one first defect in the third image through a first detection model, and to obtain at least one second defect in the near-infrared spectral image; The elimination module is used to eliminate the third defect point in the third image to obtain a fourth image, wherein the third defect point is the defect point in the third image that corresponds to the position of the second defect point.

6. The apparatus of claim 5, wherein, The image processing apparatus further includes: The filtering module is used to perform bilateral filtering on the first image to obtain a smoothed image; An extraction module is used to extract facial texture from the first image based on the difference between the smoothed image and the first image; An addition module is used to add the facial texture to the fourth image to obtain the fifth image.

Images