Image processing method and apparatus

Abstract

To provide an image processing method and apparatus capable of more excellently expressing a photographed skin color even under a light source with a different color spectrum.SOLUTION: Calculating a transformation matrix for transforming an effective range of each of a melanin component vector, a hemoglobin component vector, and a shadow component vector of the skin area under the first light source into an effective range of each of a melanin component vector, a hemoglobin component vector, and a shadow component vector under the second light source, and performing coordinate transformation on a concentration space value of the skin area under the first light source by using the transformation matrix to obtain a coordinate-transformed concentration space value; Next, the pixel value of the skin area under the first light source is updated on the basis of the coordinate-converted density space value to acquire the skin area under the second light source.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to the field of computer technology, and more particularly to an image processing method and apparatus. [Background technology]

[0002] In image capture, white balance processing is performed on the pixels output from the image sensor to compensate for differences in sensitivity of the image sensor and differences in the spectrum of the light source used. In white balance processing, when the subject includes a pure white object, the white balance of the image is achieved by multiplying the entire image by a coefficient that makes the values ​​of pixels located within the pure white subject area in the image and adjacent to each other within that area approximately the same for all color types of color filters. However, in general photography, the subject does not always include a pure white object.

[0003] Furthermore, when photographing skin and a color target illuminated by a light source of any color spectrum, differences in skin color will occur depending on the type of light source, even if an appropriate white balance is achieved depending on the light source and color correction is performed to approximate the color of the color target. To address this issue, there are imaging processing methods that reproduce skin color based on the difference between the average skin color and the captured skin color, but because skin color varies from person to person, it is difficult to obtain the optimal skin color.

[0004] Therefore, the challenge to be solved is how to better express the skin color captured under light sources with different color spectrums. Summary of the Invention

[0005] The embodiments of the present application provide an image processing method and apparatus, which can better represent the skin color captured under light sources with different color spectrums.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, which includes: an electronic device detects a skin area in an image, obtains the skin area under a first light source in the image, calculates a transformation matrix that transforms the effective range of each of the melanin component vector, the hemoglobin component vector, and the shadow component vector of the skin area under the first light source into the effective range of each of the melanin component vector, the hemoglobin component vector, and the shadow component vector under a second light source, uses the transformation matrix to perform coordinate transformation on the density space values ​​of the skin area under the first light source to obtain coordinate-transformed density space values, and then updates pixel values ​​of the skin area under the first light source based on the coordinate-transformed density space values ​​to obtain the skin area under the second light source.

[0007] In this manner, in an embodiment of the present application, the electronic device uses a transformation matrix to coordinate-transform the density space values ​​of the skin region under the first light source to obtain coordinate-transformed density space values, and then updates the pixel values ​​of the skin region under the first light source based on the coordinate-transformed density space values ​​to obtain the skin region under the second light source, thereby enabling better representation of the skin color captured under light sources with different color spectrums.

[0008] In an alternative embodiment, the electronic device further utilizes independent component analysis of skin pigment components to process density space values ​​of the skin region under the first light source to obtain a melanin component vector, a hemoglobin component vector, and a shade component vector of the skin region under the first light source, and then determines a valid range for each of the melanin component vector, the hemoglobin component vector, and the shade component vector of the skin region under the first light source, wherein the valid range for each component vector is included within a range consisting of an upper limit value and a lower limit value of the component vector.

[0009] In an optional embodiment, the electronic device further extracts RAW pixel values ​​in the skin area under the first light source, performs a separation operation on the RAW pixel values ​​to obtain red, green, and blue pixel values ​​of the skin area under the first light source, and then performs a conversion process on the red, green, and blue pixel values ​​to obtain density space values ​​of the skin area under the first light source.

[0010] In one optional embodiment, the electronic device updating pixel values ​​of the skin area under the first light source based on the coordinate-transformed density space values ​​to obtain the skin area under the second light source includes converting the coordinate-transformed density space values ​​into color space values ​​of the skin area under the second light source, and updating RAW pixel values ​​of the skin area under the first light source using the pixel values ​​of the color space values ​​to obtain the skin area under the second light source.

[0011] In one optional embodiment, the electronic device further corrects pixel values ​​of the skin area under the second light source with a preset white balance gain and preset color correction parameters to obtain the skin area under the second light source.

[0012] In an optional embodiment, the electronic device further performs demosaicing on the skin region under the second light source to obtain the skin region under the second light source.

[0013] In a second aspect, an embodiment of the present application provides an image processing device. The device includes a processing unit and a communication unit. The communication unit is used for transmitting and receiving signals / signaling. The processing unit is used for detecting a skin region in an image and acquiring a skin region under a first light source of the image. The processing unit is further used for calculating a transformation matrix for converting the valid range of each of the melanin component vector, the hemoglobin component vector, and the shade component vector of the skin region under the first light source into the valid range of each of the melanin component vector, the hemoglobin component vector, and the shade component vector under a second light source. The processing unit is further used for coordinate-transforming the density space values ​​of the skin region under the first light source using the transformation matrix to acquire coordinate-transformed density space values. The processing unit is further used for updating pixel values ​​of the skin region under the first light source based on the coordinate-transformed density space values ​​to acquire the skin region under the second light source.

[0014] Furthermore, for other optional embodiments of the image processing device in this aspect, reference can be made to the related content of the first aspect above, and detailed description thereof will not be given here.

[0015] In a third aspect, an embodiment of the present application provides an image processing device. The image processing device includes a memory and a processor. Optionally, the image processing device further includes a communication interface. The memory is used to store a computer program. The communication interface is used to receive or transmit data. The processor is used to invoke the program instructions stored in the memory.

[0016] In one alternative embodiment, the processor is used to call a computer program to perform the following operations: detect a skin region in an image and acquire the skin region under a first light source in the image; calculate a transformation matrix that converts the effective range of each of the melanin component vector, hemoglobin component vector, and shade component vector of the skin region under the first light source into the effective range of each of the melanin component vector, hemoglobin component vector, and shade component vector under a second light source; use the transformation matrix to perform coordinate transformation on the density space values ​​of the skin region under the first light source to acquire coordinate-transformed density space values; update pixel values ​​of the skin region under the first light source based on the coordinate-transformed density space values ​​to acquire the skin region under the second light source.

[0017] Furthermore, for other optional embodiments of the electronic device in this aspect, reference may be made to the related content of the first aspect above, and will not be described in detail here.

[0018] In a fourth aspect, an embodiment of the present application provides a chip. The chip includes a processor and a communication interface. The communication interface is used for receiving or transmitting data. In an alternative embodiment, the processor is configured to cause the chip to perform the following operations: detect a skin area in an image and acquire the skin area under a first light source in the image; calculate a transformation matrix that converts the valid range of each of the melanin component vector, hemoglobin component vector, and shade component vector of the skin area under the first light source into the valid range of each of the melanin component vector, hemoglobin component vector, and shade component vector under a second light source; use the transformation matrix to perform coordinate transformation on the density space values ​​of the skin area under the first light source to acquire coordinate-transformed density space values; update pixel values ​​of the skin area under the first light source based on the coordinate-transformed density space values ​​to acquire the skin area under the second light source.

[0019] Furthermore, for other optional embodiments of the chip in this aspect, reference may be made to the relevant content of the first aspect above, which will not be described in detail here.

[0020] In a fifth aspect, an embodiment of the present application provides a modular device, the modular device comprising: a communication module, a power module, a memory module, and a chip. The power module is used to supply power to the modular device. The memory module is used to store data and instructions. The communication module is used for internal communication of the modular device or for communication between the modular device and an external device. The chip is used to perform the method described in the first aspect.

[0021] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having stored thereon a computer program for use in the image processing device, the computer program including instructions for performing the method according to the first aspect.

[0022] In a seventh aspect, embodiments of the present application further provide a computer program product which, when executed on a processor, causes the method processes described in the first aspect above to be implemented. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 is a schematic diagram showing the structure of an image processing device according to an embodiment of the present application. [Figure 2] FIG. 2 is a schematic diagram showing the structure of another image processing device according to an embodiment of the present application. [Figure 3] FIG. 3 is a flowchart illustrating an image processing method according to an embodiment of the present application. [Figure 4] FIG. 4 is a schematic diagram illustrating an independent component analysis of skin pigment components according to an embodiment of the present application. [Figure 5] FIG. 5 is a schematic diagram illustrating a coordinate transformation according to an embodiment of the present application. [Figure 6] FIG. 6 is a flowchart illustrating another image processing method according to an embodiment of the present application. [Figure 7] FIG. 7 is a schematic diagram showing the structure of an image processing device according to an embodiment of the present application. [Figure 8] FIG. 8 is a schematic diagram showing the structure of a module device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0024] Hereinafter, embodiments of the present application will be described with reference to the drawings of the embodiments of the present application.

[0025] The terms "first," "second," and the like in the specification, claims, and drawings of this application are used to distinguish between different objects, not to describe a particular sequence. The terms "first," "second," and the like are used for descriptive purposes only and should not be understood to indicate or imply relative importance or the number of technical features being referred to. Thus, a feature qualified by a term such as "first," "second," or the like may expressly or imply the inclusion of one or more of the feature. In the description of the present embodiment, unless otherwise specified, "plurality" refers to two or more.

[0026] Furthermore, terms such as "comprises," "comprises," or any other variants are intended to cover and not exclude the inclusion of other elements. For example, a process, method, system, product, or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally further include other steps or units not listed, or may optionally further include other steps or units inherent to those processes, methods, systems, products, or devices.

[0027] As can be understood, in this application, "plurality" refers to two or more. The term "and / or" describes the relationship between related objects and indicates that three types of relationships exist. For example, A and / or B refers to three situations: the presence of only A, the presence of only B, and the simultaneous presence of A and B. A and B may be singular or plural. The symbol " / " generally indicates that the related objects before and after it are in an "or" relationship. Both "when" and "if" refer to the carrying out of appropriate processing under certain objective circumstances, and do not imply a time limit, nor do they require any judgmental actions to be taken when realizing the situation, nor do they imply any other limitations.

[0028] In the embodiments of the present application, terms such as "exemplary" or "for example" are used to indicate an example, illustration, or explanation. Any embodiment or solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as preferred or advantageous over other embodiments or solutions. To be clear, the use of terms such as "exemplary" or "for example" is intended to present relevant concepts in a concrete manner for ease of understanding.

[0029] Currently, there are imaging processing methods that reproduce skin color based on the difference between the average skin color and the captured skin color, but since skin color varies from person to person, it is difficult to obtain the optimal skin color.

[0030] In image capture, white balance processing is performed on the pixels output from the image sensor to compensate for differences in sensitivity among image sensors and differences in the spectrum of the light source used. In white balance processing, when a subject includes a pure white object, the white balance of the image is achieved by multiplying the entire image by a coefficient that ensures that the values ​​of adjacent pixels located within the pure white object area in the image are approximately the same for all color filter types. However, in general photography, pure white objects are not always included in the subject. Furthermore, when photographing skin and a color target illuminated by a light source of any color spectrum, differences in skin color in the image will occur depending on the light source, even if appropriate white balance is achieved depending on the light source and color correction is performed to approximate the color of the color target.

[0031] Furthermore, based on the fact that the minimum values ​​of each pigment component of human skin are nearly the same, a method has been developed in which skin is photographed under various light sources, and the minimum values ​​of the separated pigment components are determined by independent component analysis. This is then replaced with the minimum value of the pigment component of skin under a specific light source, thereby achieving color reproduction equivalent to that achieved when photographed under a specific light source. However, this method is only applicable to the specific light source, and when a light source other than the specific light source is used, there may be a difference between the converted skin color and the skin color under the target light source.

[0032] Therefore, the challenge to be solved is how to better express the skin color captured under light sources with different color spectrums.

[0033] The following describes concepts related to embodiments of the present application.

[0034] 1. Bayer array The Bayer array is a type of color pattern arrangement for image sensor color filters. In this color pattern arrangement, a color image is captured by arranging red, green, and blue filters on the image sensor, allowing each pixel to recognize only one color of light. Color filters are attached to the photodiodes (corresponding to pixels) on the surface of the image sensor in a Bayer pattern. Image light shining on the image sensor surface passes through each color filter and reaches the photodiode, which generates a voltage according to the amount of light received that passes through the attached filter color. The voltage level is converted into a digital value and output from the image sensor as Bayer array data.

[0035] 2.Independent Component Analysis (ICA) Independent component analysis is a statistical method primarily used for separating multidimensional signals, with the aim of decomposing a mixed signal into independent components.

[0036] Optionally, the electronic device applied to the embodiments of the present application may refer to various forms of user equipment (UE), access terminal, user unit, user station, mobile station, mobile station (MS), remote station, remote terminal, mobile device, user terminal, wireless communication device, user agent, or user equipment. Optionally, the electronic device may be a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication capabilities, a computing device, or other processing device connected to a wireless modem, an in-vehicle device, a wearable device, a terminal device in a 5th generation mobile communication (5G) network, or a terminal device in a future evolved public land mobile network (PLMN), etc. It may also be a digital camera, an industrial camera, a security camera, an automotive camera, a camera module, an image signal processor, and image processing software, but the embodiments of the present application are not limited thereto.

[0037] Optionally, as shown in FIG. 1, which is a schematic diagram illustrating the structure of an image processing device 100 according to an embodiment of the present application, in one embodiment, the image processing device 100 may include a processing unit 101.

[0038] The processing unit 101 is used to detect a skin area in an image and obtain a skin area under a first light source in the image.

[0039] The processing unit 101 is further used to calculate a transformation matrix that converts the effective range of each of the melanin component vector, hemoglobin component vector, and shading component vector of the skin area under the first light source into the effective range of each of the melanin component vector, hemoglobin component vector, and shading component vector under the second light source.

[0040] The processing unit 101 is further used for using a transformation matrix to coordinate-transform the density space values ​​of the skin region under the first light source, thereby obtaining coordinate-transformed density space values.

[0041] The processing unit 101 is further used to update the pixel values ​​of the skin region under the first light source according to the coordinate-transformed density space values ​​to obtain the skin region under the second light source.

[0042] Optionally, the image processing device 100 further includes a communication unit 102. The communication unit 102 is used to receive or transmit data, for example, to receive images to be processed and to transmit processed images.

[0043] In an optional embodiment, the processing unit 101 further utilizes independent component analysis of skin pigment components to process density space values ​​of the skin region under the first light source to obtain a melanin component vector, a hemoglobin component vector, and a shading component vector of the skin region under the first light source, which are used to determine an effective range of each of the melanin component vector, the hemoglobin component vector, and the shading component vector of the skin region under the first light source, where the effective range of each component vector is included in a range consisting of an upper limit value and a lower limit value of the component vector.

[0044] In an optional embodiment, the processing unit 101 is further used to extract RAW pixel values ​​in the skin area under the first light source, perform a separation operation on the RAW pixel values ​​to obtain red, green, and blue pixel values ​​of the skin area under the first light source, and perform a conversion process on the red, green, and blue pixel values ​​to obtain density space values ​​of the skin area under the first light source.

[0045] In an optional embodiment, the processing unit 101 converts the coordinate-transformed density space values ​​into color space values ​​of the skin area under the second light source, and uses pixel values ​​of the color space values ​​to update RAW pixel values ​​of the skin area under the first light source to obtain the skin area under the second light source.

[0046] In an optional embodiment, the processing unit 101 further corrects pixel values ​​of the skin area under the second light source with a preset white balance gain and preset color correction parameters to obtain the skin area under the second light source.

[0047] In an optional embodiment, the processing unit 101 further performs demosaicing on the skin region under the second light source to obtain the skin region under the second light source.

[0048] Alternatively, the operations performed by the processing unit may be performed by an optical lens unit, an image sensor unit, and an image data generation unit. The image data generation unit includes an image input unit, a dimension reduction unit, a spatial projection unit, and an image output unit. For example, FIG. 2 is a schematic diagram showing the structure of another image processing device according to an embodiment of the present application. As shown in FIG. 2, the image processing device includes an optical lens unit, an image sensor unit, and an image data generation unit, and the image data generation unit includes an image input unit, a dimension reduction unit, a spatial projection unit, and an image output unit.

[0049] The optical lens unit is used to capture an image.

[0050] The image sensor unit processes the image captured by the optical lens unit to generate RAW image data.

[0051] In the image data generation unit, the image input unit executes, by the processing unit 101 in FIG. 1, detecting a skin region in an image and acquiring a skin region under a first light source in the image. The spatial projection unit executes, by the processing unit 101 in FIG. 1, calculating a transformation matrix that converts the effective range of each of the melanin component vectors, hemoglobin component vectors, and shading component vectors of the skin region under the first light source into the effective range of each of the melanin component vectors, hemoglobin component vectors, and shading component vectors under a second light source. The spatial projection unit executes, by the processing unit 101 in FIG. 1, acquiring coordinate-transformed density space values ​​by performing coordinate transformation on the density space values ​​of the skin region under the first light source using the transformation matrix. The image output unit executes, by the processing unit 101 in FIG. 1, updating pixel values ​​of the skin region under the first light source based on the coordinate-transformed density space values ​​to acquire a skin region under a second light source.

[0052] In an alternative embodiment, the dimension reduction unit executes, as performed by the processing unit 101 in Fig. 1, obtaining a melanin component vector, a hemoglobin component vector, and a shading component vector of the skin region under the first light source by processing density space values ​​of the skin region under the first light source using independent component analysis of skin pigment components. The spatial projection unit executes, as performed by the processing unit 101 in Fig. 1, determining a valid range of each of the melanin component vector, the hemoglobin component vector, and the shading component vector of the skin region under the first light source. The valid range of each component vector is included in a range consisting of an upper limit value and a lower limit value of the component vector.

[0053] In an optional embodiment, the image input unit performs the following operations, which are performed by the processing unit 101 in FIG. 1 : extracting RAW pixel values ​​in the skin area under the first light source; performing a separation operation on the RAW pixel values ​​to obtain red, green, and blue pixel values ​​of the skin area under the first light source; and performing a conversion operation on the red, green, and blue pixel values ​​to obtain density space values ​​of the skin area under the first light source.

[0054] In one optional embodiment, the image output unit performs the steps performed by the processing unit 101 in FIG. 1 to convert the coordinate-transformed density space values ​​into color space values ​​of the skin area under the second light source, and to update the RAW pixel values ​​of the skin area under the first light source using the pixel values ​​of the color space values ​​to obtain the skin area under the second light source.

[0055] In an optional embodiment, the image output unit executes a process performed by the processing unit 101 in FIG. 1 to correct pixel values ​​of the skin area under the second light source with a preset white balance gain and preset color correction parameters to obtain the skin area under the second light source.

[0056] In an optional embodiment, the image output unit executes the demosaic process performed by the processing unit 101 in FIG. 1 on the skin region under the second light source to obtain the skin region under the second light source.

[0057] As shown in FIG. 2, RAW image data generated by an image sensor unit is input to an image data generation unit, where it is processed and skin-color-converted image data is output. As shown in FIG. 2, the image data generation unit may include an image input unit, a dimension reduction unit, a spatial projection unit, and an image output unit. The image input unit receives RAW image data, detects a skin region in the RAW image, acquires a skin region under a first light source in the RAW image, extracts pixel values ​​of the skin region under the first light source from the skin region under the first light source in the RAW image, and converts the pixel values ​​of the skin region under the first light source into density space values. The dimension reduction unit separates the density space values ​​into a melanin component vector, a hemoglobin component vector, and a shadow component vector through independent component analysis. The spatial projection unit calculates an effective range of each component vector among the melanin component vector, the hemoglobin component vector, and the shadow component vector, calculates a transformation matrix that transforms the effective range of each component vector into the effective range of each component vector under a second light source, and performs coordinate transformation on the density space values ​​using the transformation matrix to obtain coordinate-converted density space values. In other words, the spatial projection unit can geometrically transform the position and shape of the spatial domain into any spatial domain. The image output unit updates the skin region under the first light source using the coordinate-transformed density spatial values ​​and corrects the pixel values ​​of the updated skin region under the second light source using a preset white balance gain and preset color correction parameters to obtain skin-color-converted image data. Optionally, the image output unit further performs demosaic processing on the updated skin region under the second light source to obtain skin-color-converted image data.

[0058] An embodiment of the present application provides an image processing method. Figure 3 is a flowchart showing the image processing method. The method includes but is not limited to the following:

[0059] S301: The electronic device detects a skin area in an image and obtains the skin area under a first light source in the image.

[0060] In an alternative embodiment, the electronic device can detect skin regions in an image after performing shading correction on an image captured under automatic exposure control.

[0061] In an alternative embodiment, the electronic device detects a skin area in the image by an image recognition algorithm and obtains the skin area under the first light source in the image.

[0062] In an optional embodiment, the electronic device detects a skin area in an image, obtains the skin area of ​​the image under a first light source, extracts RAW pixel values ​​in the skin area under the first light source, and performs a separation operation on the RAW pixel values ​​in the skin area under the first light source according to the Bayer color filter color to obtain red (R), green (G), and blue (B) pixel values ​​of the skin area under the first light source. Next, the electronic device performs a conversion process on the red, green, and blue pixel values ​​to obtain density space values ​​of the skin area under the first light source.

[0063] Optionally, the electronic device performing a separation operation on the RAW pixel values ​​in the skin area under the first light source for each Bayer color filter color to obtain red, green, and blue pixel values ​​of the skin area under the first light source includes the electronic device performing a separation operation on the RAW pixel values ​​in the skin area under the first light source for each Bayer color filter color to obtain red, green blue (GB), green red (GR), and blue pixel values ​​of the skin area under the first light source, and obtaining the red, green, and blue pixel values ​​of the skin area under the first light source by taking the average value of the blue-green and red-green pixel values ​​as the green pixel value.

[0064] Optionally, the electronic device performing a conversion process on the red, green, and blue pixel values ​​of the skin area under the first light source to obtain density space values ​​of the skin area under the first light source includes the electronic device taking the reciprocal values ​​of the red, green, and blue pixel values ​​of the skin area under the first light source, and taking the logarithm to the base 10 of the reciprocal values ​​of the red, green, and blue pixel values ​​of the skin area under the first light source to obtain density space values ​​of the skin area under the first light source.

[0065] In an alternative embodiment, the electronic device uses independent component analysis of skin pigment components to process density space values ​​of the skin region under the first light source to obtain a melanin component vector, a hemoglobin component vector, and a shade component vector of the skin region under the first light source, and determines a valid range for each of the melanin component vector, the hemoglobin component vector, and the shade component vector of the skin region under the first light source. The valid range for each component vector is a range that excludes outliers of that component vector and is included within a range consisting of an upper limit value and a lower limit value of that component vector.

[0066] For example, FIG. 4 is a schematic diagram showing an independent component analysis of skin pigment components according to an embodiment of the present application. In the coordinate system shown in FIG. 4, the reciprocal values ​​of three pixel values, R, G, and B, are respectively taken, and the logarithm of the three reciprocal values ​​with base 10 is taken, and the three logarithm values ​​are set as the x-axis, y-axis, and z-axis of the coordinate system. The logarithm of the reciprocal value of the B pixel value with base 10 (i.e., -log 10 B) is the x-axis, and the logarithm of the reciprocal of the G pixel value with base 10 (i.e., -log 10 G) is the y-axis, and the logarithm of the reciprocal of the R pixel value with base 10 (i.e., -log 10 R) is the z-axis. In the coordinate system, the melanin component vector of the skin region under the first light source is represented by a solid line, the hemoglobin component vector of the skin region under the first light source is represented by a dashed-dotted line, and the shading component vector of the skin region under the first light source is represented by a dashed line. The upper and lower limit values ​​of each of the melanin component vector, hemoglobin component vector, and shading component vector are the two endpoints of the corresponding component vector.

[0067] In this embodiment, if there are outliers in the skin pigment components, the melanin component vector, hemoglobin component vector, and shading component vector obtained by the electronic device using independent component analysis of the skin pigment components will also contain outliers. In response to this, the electronic device can reduce the influence of disturbances on the results obtained by the independent component analysis of the skin pigment components by determining an effective range for each of the melanin component vector, hemoglobin component vector, and shading component vector of the skin region under the first light source, where the effective range for each component vector is a range that excludes outliers in that component vector and is included within a range defined by the upper and lower limits of that component vector.

[0068] S302: The electronic device calculates a transformation matrix that converts the effective range of each of the melanin component vector, hemoglobin component vector, and shading component vector of the skin area under the first light source into the effective range of each of the melanin component vector, hemoglobin component vector, and shading component vector under the second light source.

[0069] S303: The electronic device uses a transformation matrix to transform the coordinates of the density space values ​​of the skin region under the first light source, thereby obtaining coordinate-transformed density space values.

[0070] In one alternative embodiment, the electronic device multiplies the density space values ​​of the skin region under the first light source by a transformation matrix to obtain coordinate-transformed density space values.

[0071] For example, Fig. 5 is a schematic diagram showing coordinate transformation according to an embodiment of the present application. As shown in Fig. 5, the range frame of the first light source is composed of eight vertices each consisting of a combination of upper and lower limit values ​​of the effective range of each of the melanin component vector, hemoglobin component vector, and shading component vector of the skin region under the first light source, and the range frame of the second light source is composed of eight vertices each consisting of a combination of upper and lower limit values ​​of the effective range of each of the melanin component vector, hemoglobin component vector, and shading component vector of the skin region under the second light source.

[0072] The coordinate-transformed density space values ​​are density space values ​​of the skin region under the second light source. The electronic device obtains the density space values ​​of the skin region under the second light source by using a transformation matrix to coordinate-transform the density space values ​​of the skin region under the first light source.

[0073] S304: The electronic device updates the pixel values ​​of the skin area under the first light source based on the coordinate-transformed density space values ​​to obtain the skin area under the second light source.

[0074] In an optional embodiment, the electronic device converts the coordinate-transformed density space values ​​into color space values ​​of the skin area under the second light source, and updates the RAW pixel values ​​of the skin area under the first light source using pixel values ​​of the color space values ​​to obtain the skin area under the second light source.

[0075] In an alternative embodiment, the color space values ​​of the skin area under the second light source are coordinate-transformed red, green, and blue pixel values ​​of the skin area under the first light source. As can be seen from S301, the electronic device obtains the density space values ​​of the skin area under the first light source by performing a conversion process of taking the reciprocal and logarithmic values ​​of the red, green, and blue pixel values ​​of the skin area under the first light source. That is, the electronic device can convert the coordinate-transformed density space values ​​into color space values ​​of the skin area under the second light source based on the relationship between the red, green, and blue pixel values ​​of the skin area under the first light source and the density space values ​​of the skin area under the first light source.

[0076] In an optional embodiment, the electronic device further corrects pixel values ​​of the skin region under the second light source with a preset white balance gain and preset color correction parameters.

[0077] White balance is an index that indicates the accuracy of the white color generated by mixing the three primary colors of red, green, and blue on a monitor. Because the color temperature varies depending on the light source, adjusting the white balance gain corrects the color temperature and restores the color of the subject in the image, allowing for accurate color expression.

[0078] Color correction is a method of correcting the colors of an image based on red, green, and blue color theory.

[0079] Optionally, the electronic device further demosaices the skin region under the second light source.

[0080] In the demosaic process, since each pixel in the Bayer array data has only one of the pixel values ​​of the filter color R, G, or B, the pixel values ​​of the missing filter color of a pixel are generated using the pixel values ​​of the surrounding pixels of the pixel. For example, if a pixel has an R pixel value and the pixel values ​​of the missing filter colors of the pixel are G pixel values ​​and B pixel values, the G pixel values ​​and B pixel values ​​of the surrounding pixels of the pixel can be used to fill in the G pixel values ​​and B pixel values ​​of the pixel.

[0081] In this way, the electronic device detects a skin area in an image, obtains the skin area under a first light source in the image, and obtains coordinate-transformed density space values ​​by using a transformation matrix to transform the density space values ​​of the skin area under the first light source. Then, based on the coordinate-transformed density space values, the electronic device updates the pixel values ​​of the skin area under the first light source to obtain the skin area under a second light source. In this way, the electronic device can better represent the skin color of any subject in the image under any light source.

[0082] Referring to Figure 6, Figure 6 is a flowchart illustrating another image processing method according to an embodiment of the present application, which includes but is not limited to the following steps:

[0083] S601: The electronic device receives RAW image data.

[0084] The electronic device receives the RAW image generated by the image sensor.

[0085] S602: The electronic device detects a skin area in the RAW image and obtains the skin area under the first light source in the image.

[0086] For the content related to S602, please refer to the related description of step S301 in the image processing method shown in FIG. 3 above, and a detailed description will not be given.

[0087] S603: The electronic device extracts pixel values ​​of the skin region under the first light source from the skin region under the first light source in the RAW image.

[0088] For the relevant content of S603, please refer to the relevant description of step S301 in the image processing method shown in FIG.

[0089] S604: The electronic device separates the pixel values ​​of the extracted skin region under the first light source according to the color of the color filter to obtain separated pixel values.

[0090] Optionally, step S604 may specifically include the following content: The electronic device separates the extracted pixel values ​​of the skin region under the first light source into three color pixel values ​​of red, green, and blue according to the color of the Bayer color filter.

[0091] S605: The electronic device converts the separated pixel values ​​into density space values.

[0092] For the relevant content of S605, please refer to the relevant description of step S301 in the image processing method shown in FIG.

[0093] S606: The electronic device separates the density spatial values ​​into a melanin component vector, a hemoglobin component vector, and a shade component vector by independent component analysis.

[0094] For the relevant content of S606, reference can be made to the relevant description of step S301 in the image processing method shown in FIG.

[0095] S607: The electronic device obtains the effective range of each component from each of the melanin component vector, the hemoglobin component vector, and the shade component vector.

[0096] The effective ranges of the melanin component vector, hemoglobin component vector, and shade component vector are each included within the range defined by the upper and lower limits of each component vector.

[0097] S608: The electronic device calculates a transformation matrix that converts the effective range of each of the melanin component vector, the hemoglobin component vector, and the shadow component vector into the effective range of each of the melanin component vector, the hemoglobin component vector, and the shadow component vector under the second light source.

[0098] S609: The electronic device performs coordinate transformation on the density space values ​​using the transformation matrix to obtain coordinate-transformed density space values.

[0099] For the relevant content of S609, please refer to the relevant description of FIG. 5 mentioned above.

[0100] S6010: The electronic device converts the coordinate-transformed density space values ​​into color space values.

[0101] For the relevant content of S6010, please refer to the relevant description of step S304 in the image processing method shown in FIG.

[0102] S6011: The electronic device updates the pixel values ​​of the skin region under the first light source using the pixel values ​​of the color space values.

[0103] For the relevant content of S6011, reference can be made to the relevant description of step S304 in the image processing method shown in FIG.

[0104] S6012: The electronic device corrects the pixel values ​​of the updated skin region under the second light source using a preset white balance gain.

[0105] Optionally, the preset white balance gain is a white balance gain determined to better reproduce skin tones.

[0106] For the relevant content of S6012, reference can be made to the relevant description of step S304 in the image processing method shown in FIG.

[0107] S6013: The electronic device performs demosaic processing on the updated skin region under the second light source.

[0108] For the content related to S6013, please refer to the related description of step S304 in the image processing method shown in FIG.

[0109] S6014: The electronic device corrects the pixel values ​​of the updated skin region under the second light source using the preset color correction parameters.

[0110] For the relevant content of S6014, reference can be made to the relevant description of step S304 in the image processing method shown in FIG.

[0111] By executing steps S601 to S6014, the electronic device can obtain image data after skin color conversion and selectively further output the image data after skin color conversion.

[0112] Referring to Fig. 7, Fig. 7 is a schematic diagram showing the structure of an image processing device 700 according to an embodiment of the present application. The image processing device 700 may be a terminal device or a network device. The image processing device 700 may include a memory 701 and a processor 702. Optionally, the image processing device 700 may further include a communication interface 703. The memory 701, the processor 702, and the communication interface 703 are connected via one or more communication buses. The communication interface 703 is controlled by the processor 702 and is used to transmit and receive information.

[0113] Memory 701 may include read only memory (ROM) and random access memory (RAM) and provides instructions and data to processor 702. A portion of memory 701 may further include non-volatile random access memory.

[0114] The communication interface 703 is used to receive or transmit data.

[0115] The processor 702 may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. A general-purpose processor may be a microprocessor, or alternatively, the processor 702 may be any conventional processor, etc.

[0116] The memory 701 is used to store program instructions.

[0117] The processor 702 is used to access program instructions stored in the memory 701 .

[0118] In an alternative embodiment, the processor 702 is used to perform the following operations when calling the computer program: Detect a skin region in an image and obtain a skin region under a first light source in the image; Calculate a transformation matrix that converts the valid range of each of the melanin component vector, hemoglobin component vector, and shade component vector of the skin region under the first light source into the valid range of each of the melanin component vector, hemoglobin component vector, and shade component vector under a second light source; Use the transformation matrix to perform coordinate transformation on the density space values ​​of the skin region under the first light source to obtain coordinate-transformed density space values; Update pixel values ​​of the skin region under the first light source based on the coordinate-transformed density space values ​​to obtain the skin region under the second light source.

[0119] For other embodiments of the image processing device, reference may be made to the relevant content of the above method embodiments, and will not be described in detail here.

[0120] Since the embodiments of the present application and the above method embodiments are based on the same idea and achieve the same technical effects, the specific principles can be referred to the description of the above method embodiments and will not be described in detail here.

[0121] The embodiments of the present application further provide a chip, which can perform the steps related to the image processing device in the above method embodiments, and which includes a processor and a communication interface, which is used to receive or transmit data.

[0122] In one embodiment, the chip performs the relevant steps in the above method embodiment. The processor is configured to cause the chip to perform the following operations: detect a skin area in an image and obtain a skin area under a first light source in the image; calculate a transformation matrix that converts the valid range of each of the melanin component vector, hemoglobin component vector, and shade component vector of the skin area under the first light source into the valid range of each of the melanin component vector, hemoglobin component vector, and shade component vector under a second light source; use the transformation matrix to perform coordinate transformation on the density space values ​​of the skin area under the first light source to obtain coordinate-transformed density space values; update pixel values ​​of the skin area under the first light source based on the coordinate-transformed density space values ​​to obtain the skin area under the second light source.

[0123] For other embodiments of the chip, reference may be made to the relevant content of the above method embodiments, and will not be described in detail here.

[0124] Since the embodiments of the present application and the above method embodiments are based on the same idea and achieve the same technical effects, the specific principles can be referred to the description of the above method embodiments and will not be described in detail here.

[0125] For each device or product applied to or integrated into a chip, each module included therein may be implemented by hardware such as a circuit, or at least a portion of the module may be implemented by a software program executed by a processor integrated within the chip, with the remaining portion of the module (if any) being implemented by hardware such as a circuit.

[0126] As shown in Figure 8, Figure 8 is a schematic diagram showing the structure of a module device according to an embodiment of the present application. The module device 800 is used to perform steps related to the image processing device in the above method embodiment. The module device 800 includes a communication module 801, a power module 802, a storage module 803, and a chip 804.

[0127] The power supply module 802 is used to supply power to the module device. The memory module 803 is used to store data and instructions. The communication module 801 is used for internal communication of the module device or for communication between the module device and external devices. The chip 804 is used to execute the methods performed by the image processing device in the above method embodiments.

[0128] For the embodiment of the module device, reference may be made to the relevant content of the above method embodiment, and no detailed description will be given here.

[0129] Since the embodiments of the present application and the above method embodiments are based on the same idea and achieve the same technical effects, the specific principles can be referred to the description of the above method embodiments and will not be described in detail here.

[0130] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, realizes the method flow of the above method embodiment.

[0131] An embodiment of the present application further provides a computer program product, which, when executed on a processor, realizes the method flows of the above method embodiments.

[0132] Each module / unit included in each device and product described in the above embodiments may be a software module / unit or a hardware module / unit, or may be partly software modules / units and partly hardware modules / units. For example, for each device or product applied to or integrated into a chip, each module / unit included therein may be implemented by hardware such as a circuit, or at least part of the module / unit may be implemented by a software program executed by a processor integrated in the chip, and the remaining part of the module / unit (if any) may be implemented by hardware such as a circuit. For each device or product applied to or integrated into a chip module, each module / unit included therein may be implemented by hardware such as a circuit, and different modules / units may be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least part of the module / unit may be implemented by a software program executed by a processor integrated in the chip module, and the remaining part of the module / unit (if any) may be implemented by hardware such as a circuit. For each device or product applied to or integrated into a terminal, the modules / units included therein may be implemented by hardware such as circuits, and different modules / units may be located in the same component (e.g., chip, circuit module, etc.) or different components within the terminal, or at least a part of the modules / units may be implemented by a software program executed on a processor integrated within the terminal, and the remaining part of the modules / units (if any) may be implemented by hardware such as circuits.

[0133] It should be noted that for simplicity, the above method embodiments are expressed as a combination of a series of operations. However, it should be understood by those skilled in the art that the present application is not limited to the order of operations described, and that some operations may be performed in other orders or simultaneously based on the present application. It should also be understood by those skilled in the art that the embodiments described in the specification are preferred embodiments, and that such operations and modules are not necessarily required for the present application.

[0134] The descriptions of the embodiments of the present application may be mutually referenced. Each embodiment has its own focus. For parts of an embodiment that are not described in detail, reference may be made to the relevant descriptions of other embodiments. For convenience and conciseness of description, for example, the functions and operations performed by each apparatus and device according to the embodiments of the present application may be referred to the relevant descriptions of the method embodiments of the present application, and each method embodiment and each apparatus embodiment may be mutually referenced, combined, or cited.

[0135] Finally, the above embodiments are only used to explain the technical solutions of the present application, and do not limit the technical solutions of the present application. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand the following: Those skilled in the art may still amend the technical solutions described in the above embodiments, and may make equivalent substitutions for some or all of these technical features, and these amendments or substitutions will not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. 1. An image processing method, comprising: Detecting a skin area in an image to obtain a skin area under a first light source in the image; calculating a transformation matrix that transforms the effective range of each of the melanin component vector, hemoglobin component vector, and shading component vector of the skin region under the first light source into the effective range of each of the melanin component vector, hemoglobin component vector, and shading component vector under a second light source; Using the transformation matrix, coordinate-transform the density space values ​​of the skin region under the first light source to obtain coordinate-transformed density space values; updating pixel values ​​of the skin region under the first light source based on the coordinate-transformed density space values ​​to acquire the skin region under the second light source; Including, An image processing method comprising:

2. The image processing method includes: obtaining the melanin component vector, the hemoglobin component vector, and the shade component vector of the skin region under the first light source by processing density space values ​​of the skin region under the first light source using an independent component analysis of skin pigment components; determining an effective range of each of the melanin component vector, the hemoglobin component vector, and the shade component vector of the skin region under the first light source, wherein the effective range of each component vector is included within a range consisting of an upper limit value and a lower limit value of the component vector; further comprising:

2. The image processing method according to claim 1.

3. The image processing method includes: Extracting raw pixel values ​​within a skin region under the first light source; performing a separation operation on the raw pixel values ​​to obtain red, green, and blue pixel values ​​of the skin region under the first light source; performing a conversion process on the red, green, and blue pixel values ​​to obtain density spatial values ​​of the skin region under the first light source; further comprising:

3. The image processing method according to claim 2.

4. updating pixel values ​​of the skin region under the first light source based on the coordinate-transformed density space values ​​to acquire the skin region under the second light source; converting the coordinate-transformed density space values ​​into color space values ​​of the skin region under the second light source; updating raw pixel values ​​of the skin region under the first light source using pixel values ​​of the color space values ​​to obtain the skin region under the second light source; Including, 2. The image processing method according to claim 1.

5. The image processing method includes: and further comprising correcting pixel values ​​of the skin region under the second light source with a preset white balance gain and preset color correction parameters to obtain the skin region under the second light source.

2. The image processing method according to claim 1.

6. The image processing method includes: further comprising performing demosaic processing on the skin region under the second light source to obtain the skin region under the second light source.

6. The image processing method according to claim 5.

7. An image processing device comprising a processing unit and a communication unit, the communication unit is used for transmitting and receiving signals / signaling; the processing unit is used to detect a skin area in an image and obtain a skin area under a first light source in the image; the processing unit is further used to calculate a transformation matrix that transforms the effective range of each of the melanin component vector, the hemoglobin component vector, and the shading component vector of the skin region under the first light source into the effective range of each of the melanin component vector, the hemoglobin component vector, and the shading component vector under a second light source; the processing unit is further configured to use the transformation matrix to transform the density space values ​​of the skin region under the first light source into coordinates, thereby obtaining coordinate-transformed density space values; the processing unit is further used to update pixel values ​​of the skin region under the first light source based on the coordinate-transformed density space values ​​to obtain the skin region under the second light source.

1. An image processing device comprising:

8. An image processing device comprising a processor and a memory, The processor and the memory are connected to each other, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to perform the image processing method according to any one of claims 1 to 6.

1. An image processing device comprising:

9. A chip comprising a processor and a communication interface, The processor is configured to cause the chip to perform the image processing method according to any one of claims 1 to 6. A chip characterized by:

10. A modular device comprising a communication module, a power module, a storage module, and a chip, the power supply module is used to supply power to the module device; the storage module is used to store data and instructions; The communication module is used for internal communication within the module device or for communication between the module device and an external device; The chip is used to execute the image processing method according to any one of claims 1 to 6. A modular device characterized by:

11. 1. A computer-readable storage medium, comprising: a computer program stored in the computer-readable storage medium, the computer program causing the image processing device to execute the image processing method according to any one of claims 1 to 6 when executed by the image processing device; A computer-readable storage medium comprising:

Images