Light source estimation method, image processing method, imaging device, and storage medium

Abstract

This application provides a light source estimation method, an image processing method, an imaging device, and a storage medium. [Solution] The light source estimation method includes: determining the pigment density distribution determination range for multiple measurement light sources separately using multiple measurement light sources of different types to obtain multiple pigment density distribution determination ranges corresponding to multiple measurement light sources; obtaining a target image of a subject under the estimated target light source; determining the skin region of the target image; determining the skin pixel distribution points in density space for the target image; projecting the skin pixel distribution points of the target image onto a unit plane along the direction of the shading component vector to obtain pigment component distribution points on the unit plane for the target image; determining the proportion of pixels located within each pigment density distribution determination range for the target image based on the pigment component distribution points of the target image; and determining the type of estimated target light source based on the proportion of pixels located within each pigment density distribution determination range for the target image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a light source estimation method, an image processing method, an imaging device, and a storage medium.

Background Art

[0002] When taking a photo or video, color deviation may occur in the captured image due to light irradiation conditions. In order to reproduce the original color of an object and make the image look more realistic and natural, white balance processing is performed on the image. Since different irradiation light sources have different effects on the shooting effect, it is important to distinguish different irradiation light sources in order to achieve appropriate white balance processing.

[0003] Currently, usually, the light source is estimated based on the color difference between the pre-calculated average skin color and any captured skin color. However, estimation errors occur due to differences in individual skin colors, and the light source estimation is not accurate.

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to solve the above-described technical problems, this application provides a light source estimation method, an image processing method, an imaging device, and a storage medium.

Means for Solving the Problems

[0005] A first aspect of the present invention provides a method for estimating a light source based on the distribution of pigment components in the skin. The light source estimation method includes: using multiple measuring light sources of different types as light sources, separately determining the pigment density distribution determination range for multiple measuring light sources to obtain multiple pigment density distribution determination ranges corresponding to multiple measuring light sources, wherein each measuring light source corresponds to one pigment density distribution determination range; obtaining a target image of a subject under the light source to be estimated; determining the skin region of the target image; converting the pixel values ​​of the skin region of the target image into density values ​​to obtain skin pixel distribution points in density space for the target image; projecting the skin pixel distribution points of the target image onto a unit plane along the direction of the shading component vector to obtain pigment component distribution points on the unit plane for the target image; determining the proportion of pixels located within each pigment density distribution determination range for the target image based on the pigment component distribution points of the target image and the pigment density distribution determination range corresponding to each measuring light source; and determining the type of light source to be estimated based on the proportion of pixels located within each pigment density distribution determination range for the target image.

[0006] The light source estimation method of this invention allows for the accurate determination of the type of light source to be estimated by first determining the pigment density distribution determination range of multiple known types of measurement light sources, determining the pigment component distribution points of a target image under the light source to be estimated, and then comparing the pigment component distribution points of the target image with the pigment density distribution determination range of the measurement light source.

[0007] A second aspect of the present invention provides an image processing method. The image processing method includes determining the type of light source to be estimated by the light source estimation method based on the skin pigment component distribution described in the first aspect; determining a target white balance gain and / or target color correction value based on the determined type of light source to be estimated; and processing an image acquired under the light source to be estimated based on the target white balance gain and / or target color correction value.

[0008] The light source estimation method provided by the embodiment of the present invention can accurately determine the type of light source to be estimated, and furthermore, can improve the image processing quality by processing an image acquired under the estimated light source using the white balance gain and / or color correction value corresponding to the estimated light source.

[0009] A third aspect of the present application provides an imaging device. The imaging device includes an image acquisition unit, a determination range calculation unit, and a light source estimation unit. The image acquisition unit is used to acquire a target image of a subject under an estimated target light source and to determine the skin region of the target image. The determination range calculation unit is used to determine the pigment density distribution determination range of multiple measurement light sources separately, using multiple measurement light sources of different types, in order to obtain multiple pigment density distribution determination ranges corresponding to multiple measurement light sources, with each measurement light source corresponding to one pigment density distribution determination range. The light source estimation unit converts the pixel values ​​of the skin region of the target image into density values ​​to obtain skin pixel distribution points in density space for the target image, projects the skin pixel distribution points of the target image onto a unit plane along the direction of the shading component vector to obtain pigment component distribution points of the target image, determines the proportion of pixels located within each pigment density distribution determination range for the target image based on the pigment component distribution points of the target image and the pigment density distribution determination range corresponding to each measurement light source, and determines the type of estimated target light source based on the proportion of pixels located within each pigment density distribution determination range for the target image.

[0010] The imaging device provided in the third embodiment has features corresponding to the light source estimation method provided in the first embodiment, and can achieve the same or corresponding beneficial effects as the light source estimation method provided in the first embodiment, which will not be described in detail here.

[0011] A fourth aspect of the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is loaded into a processor and executed, the light source estimation method described in the first aspect and / or the image processing method described in the second aspect are realized.

[0012] The computer-readable storage medium provided in the fourth embodiment is used to implement the light source estimation method described in the first embodiment and / or the image processing method described in the second embodiment, and can achieve the same or corresponding beneficial effects as the light source estimation method and / or the image processing method provided in the first embodiment, and will not be described in detail here. [Brief explanation of the drawing]

[0013] To more clearly illustrate the technical solutions of the embodiments of this application, the necessary drawings for the embodiments are briefly introduced below. Clearly, the drawings described are of some embodiments of this application, and those skilled in the art can obtain other drawings from these without any creative effort. [Figure 1] This is a flowchart of a method for estimating a light source based on the distribution of skin pigment components, according to one embodiment of the present invention. [Figure 2] This is a further subflowchart of step S1 in Figure 1. [Figure 3] This is a schematic diagram showing the structure of a pixel unit in an image obtained in one embodiment of the present invention. [Figure 4] This is a schematic diagram showing the structure of a pixel unit in an image obtained in another embodiment of the present invention. [Figure 5] Figure 2 shows a further subflowchart of step S13. [Figure 6] This is a schematic diagram showing the distribution of skin pixels in density space for a sample image relating to one embodiment of the present invention. [Figure 7] This is a schematic diagram showing a sample image relating to one embodiment of the present invention, in which the distribution points of skin pixels in density space are projected onto a unit plane along the direction of the shading component vector. [Figure 8] Figure 2 shows a further subflowchart of step S16. [Figure 9] Figure 8 shows a further subflowchart of step S161. [Figure 10] Figure 9 shows a further subflowchart of step S1613. [Figure 11]Schematic diagram showing hemoglobin components and melanin components on the unit plane according to one embodiment of the present application. [Figure 12] It is a further sub - flowchart of step S16133 in FIG. 10. [Figure 13] Schematic diagram showing the effective range of dye concentration distribution according to one embodiment of the present application. [Figure 14] It is another further sub - flowchart of step S1613 in FIG. 9. [Figure 15] Schematic diagram showing the first boundary point, the second boundary point, the third boundary point, and the fourth boundary point according to one embodiment of the present application. [Figure 16] It is a further sub - flowchart of step S162 in FIG. 8. [Figure 17] Schematic diagram showing the effective range of dye concentration distribution and the determination range of dye concentration distribution of a plurality of sample images according to one embodiment of the present application. [Figure 18] Schematic diagram showing the determination ranges of dye concentration distribution corresponding to a plurality of measurement light sources and the effective ranges of dye concentration distribution corresponding to each measurement light source according to one embodiment of the present application. [Figure 19] Schematic diagram showing the determination ranges of dye concentration distribution corresponding to a plurality of measurement light sources and the second dye component distribution points of the target image according to one embodiment of the present application. [Figure 20] It is a further sub - flowchart of step S7 in FIG. 1. [Figure 21] Schematic diagram showing the determination ranges of dye concentration distribution corresponding to a plurality of measurement light sources and the second dye component distribution points of the target image according to another embodiment of the present application. [Figure 22] It is another further sub - flowchart of step S1613 in FIG. 9. [Figure 23] Schematic diagram showing the effective range of dye concentration distribution according to another embodiment of the present application. [Figure 24] It is a further sub - flowchart of step S6 in FIG. 1. [Figure 25]This is a schematic diagram showing the pigment concentration distribution determination range, hemoglobin concentration distribution effective range, melanin concentration distribution effective range, and second pigment component distribution point corresponding to a sample image on a unit plane before step S61 is performed, relating to one embodiment of the present application. [Figure 26] This is a schematic diagram showing the pigment concentration distribution determination range, hemoglobin concentration distribution effective range, melanin concentration distribution effective range, and second pigment component distribution point corresponding to a sample image on a unit plane after step S61 and before step S62, relating to one embodiment of the present application. [Figure 27] This is a schematic diagram showing the pigment concentration distribution determination range, hemoglobin concentration distribution effective range, melanin concentration distribution effective range, and second pigment component distribution point corresponding to a sample image on a unit plane after step S62 and before step S63, relating to one embodiment of the present application. [Figure 28] This is a schematic diagram showing the pigment concentration distribution determination range, hemoglobin concentration distribution effective range, melanin concentration distribution effective range, and second pigment component distribution point corresponding to a sample image on a unit plane after step S63 and before step S64, relating to one embodiment of the present application. [Figure 29] This is a schematic diagram showing the updated dye concentration distribution determination range and the updated second dye component distribution point on the unit plane after step S64 has been performed, relating to one embodiment of the present application. [Figure 30] This is a schematic diagram showing the structure of an imaging device relating to one embodiment of the present application. [Figure 31] This is a schematic diagram showing the structure of an imaging device relating to another embodiment of the present invention. [Modes for carrying out the invention]

[0014] The technical solutions of the embodiments of this application will be described clearly and completely below with reference to the drawings of the embodiments of this application. Clearly, the embodiments described are only some, and not all, embodiments of this application. All other embodiments that a person skilled in the art could obtain without creative effort based on the embodiments of this application are within the scope of protection of this application.

[0015] The terms “first,” “second,” “third,” “fourth,” “fifth,” “sixth,” “seventh,” and “eighth” used in the specification, claims, and drawings of this application are used solely to distinguish the subjects being described and have no order or technical meaning whatsoever.

[0016] In the specification of this application, the term "and / or" simply describes the relationship between related objects and indicates that there are three types of relationships. For example, in the case of A and / or B, it indicates three situations: A exists alone, A and B exist simultaneously, or B exists alone. In this specification, the symbol " / " generally indicates that the preceding and following related objects have an "or" relationship. For example, in the case of A / B, it indicates A or B. In the embodiments of this application, "multiple" refers to two or more.

[0017] The “Examples” as used herein mean that any particular features, structures, or characteristics described in conjunction with an Example may be included in at least one Example of this Application. The term “Examples” as used anywhere in the Specification does not necessarily refer to the same Example, nor do they represent mutually exclusive, independent, or selectable Examples. Those skilled in the art will understand, either expressly or implicitly, that the Examples described herein can be combined with other Examples.

[0018] In related technologies, when pixel values ​​of a skin image are converted to density values ​​and then independent component analysis is performed, the density values ​​can be separated into melanin components, hemoglobin components, and shading components. The distribution area of ​​melanin and hemoglobin pigment components in a skin image is determined by the color spectrum of the imaging device and the illumination light source (see, for example, Patent Document No. 2015-144420, "Image Processing Device, Image Processing System, Image Processing Method, Program and Storage Medium"). When the same imaging device and the same light source are used, even when photographing different people (for example, different Japanese subjects), the distribution area of ​​melanin and hemoglobin pigment components in the skin appears in almost the same position. That is, the distribution area of ​​pigment is mainly influenced by the type of light source and the imaging device, and has little relationship with the subject. By utilizing this characteristic, this application enables highly accurate estimation of the light source by analyzing the distribution state of melanin and hemoglobin pigment components contained in the skin portion of a captured image.

[0019] The following describes a method for estimating a light source based on the distribution of skin pigment components, as provided by the embodiments of this application.

[0020] Referring to Figure 1, Figure 1 is a flowchart of a method for estimating a light source based on the distribution of skin pigment components according to one embodiment of the present invention. As shown in Figure 1, the method for estimating a light source based on the distribution of skin pigment components includes the following steps S1 to S7.

[0021] S1, using multiple light sources of different types as light sources, the dye concentration distribution determination range for each of the multiple light sources is determined separately, and multiple dye concentration distribution determination ranges corresponding to the multiple light sources are obtained, with each light source corresponding to one dye concentration distribution determination range.

[0022] S2: Obtain the target image of the subject under the estimated target light source.

[0023] S3: Determine the skin region in the target image.

[0024] S4, the pixel values ​​of the skin region in the target image are converted to density values ​​to obtain the skin pixel distribution points in density space for the target image.

[0025] S5, the skin pixel distribution points of the target image are projected onto the unit plane along the direction of the shading component vector to obtain the pigment component distribution points on the unit plane for the target image.

[0026] S6. Based on the pigment component distribution points of the target image and the corresponding pigment density distribution determination range for each measurement light source, the proportion of pixels located within each pigment density distribution determination range in the target image is determined.

[0027] S7. Based on the proportion of pixels located within each pigment density distribution determination range in the target image, the estimated target light source type is determined.

[0028] The light source used for measurement is a known type of light source. For example, the light source used for measurement can be D65, U30, TL84, CWF, A, Hz, etc.

[0029] If the same imaging device and light source are used, the pigment distribution will be similar even for different subjects' skin. Therefore, by pre-determining the pigment density distribution determination range for multiple known types of measurement light sources, determining the pigment component distribution points in the target image under the estimated target light source, and comparing the pigment component distribution points in the target image with the pigment density distribution determination range of the measurement light source, the type of estimated target light source can be accurately determined.

[0030] Figure 2 is a further subflowchart of step S1 in Figure 1. In some embodiments, determining the dye concentration distribution range in step S1, as shown in Figure 2, includes the following steps S11 to S16.

[0031] S11. Multiple sample images corresponding to multiple samples are obtained under the same measurement light source.

[0032] S12: Determine the skin area for each sample image.

[0033] S13. Convert the pixel values ​​of the skin region in each sample image into density values ​​to obtain the skin pixel distribution points in density space for each sample image.

[0034] S14. The skin pixel distribution points for each sample image are projected onto the unit plane along the direction of the shading component vector to obtain the pigment component distribution points on the unit plane for each sample image.

[0035] In S15, based on the pigment component distribution points for each sample image, the pigment components are separated to obtain hemoglobin and melanin components.

[0036] S16. Based on the hemoglobin and melanin components of each sample image, the pigment density distribution determination range on the unit plane is determined for the measurement light source.

[0037] The light source estimation method provided in this embodiment only requires attention to the pigment components of the skin and does not require shadow components. Therefore, by removing the shadow components beforehand and further separating the pigment components to obtain melanin and hemoglobin components, the separation process can be simplified. Since the shadow components have been removed, interference from the shadow components can be eliminated, improving the separation accuracy of melanin and hemoglobin components, and further improving the accuracy of the determined pigment concentration distribution range.

[0038] In some embodiments, step S12 allows for the identification of skin color by thresholding of the color space, and further determination of skin regions. In other embodiments, skin regions can be identified by an instrumental learning method. For example, a training model is generated by training a learning model such as a Support Vector Machine (SVM), a random forest, or a convolutional neural network to learn skin features, and the skin regions of a sample image are identified by this training model.

[0039] In some embodiments, the sample image obtained in step S11 is the original image data with automatic exposure and shadow correction applied, and no processing such as white balance, color correction, or gamma correction has been performed. The pixel values ​​of the sample image can be understood to be close to the original pixel values.

[0040] In some examples, when separating pigment components based on the pigment component distribution points for each sample image, the pigment components can be separated into melanin and hemoglobin components by independent component analysis (ICA). In other examples, the pigment components can be separated into melanin and hemoglobin components by other component analysis methods.

[0041] Referring to Figures 3 and 4, Figure 3 is a schematic diagram showing the structure of a pixel unit 10 in an image obtained in one embodiment of the present application. Figure 4 is a schematic diagram showing the structure of a pixel unit 10 in an image obtained in another embodiment of the present application. In some embodiments, as shown in Figure 3, the sample image includes a plurality of pixel units 10, each pixel unit 10 includes a plurality of subpixels 11, and the pixel values ​​of the skin regions of the sample image are the pixel values ​​of the subpixels 11.

[0042] Figure 5 is a further subflowchart of step S13 in Figure 2. In some embodiments, converting the pixel values ​​of the skin region for each sample image in step 13 to density values, as shown in Figure 5, includes the following steps S131 and S132.

[0043] S131: Obtain the pixel values ​​of all subpixels 11 in the skin region for each sample image.

[0044] In S132, the pixel values ​​of each sub-pixel 11 for each pixel unit 10 are determined, and the logarithm of the reciprocal of the pixel values ​​of each sub-pixel 11 for each pixel unit 10 is calculated to obtain the density values ​​corresponding to the pixel values ​​of each sub-pixel 11 for each pixel unit 10.

[0045] The pixel value reflects the light beam reflected from the object surface and entering the imaging lens, and it is directly proportional to the reflectance of the object surface. According to the Lambert-Beer law, the logarithmic value of the reciprocal of the reflectance is directly proportional to the pigment content. By logarithmically transforming the reciprocal of the pixel value, the pixel value can be converted to a value directly proportional to the pigment content. This conversion establishes a linear relationship between the pixel value and the pigment content, which is therefore advantageous for subsequent linear separation methods (e.g., ICA) for separating pigment components.

[0046] In some embodiments, determining the pixel values ​​of various subpixels 11 for each pixel unit 10 includes determining that the pixel value of a subpixel 11 of any one type is one if any one pixel unit 10 has one such subpixel 11, and taking the average value of the pixel values ​​of at least two such subpixels 11 as the pixel value of that subpixel 11 if any one pixel unit 10 has at least two such subpixels 11.

[0047] The average of the pixel values ​​of at least two such sub-pixels 11 can be the arithmetic mean, the weighted mean, and so on.

[0048] For example, if a pixel unit 10 has one subpixel of type 1 and the pixel value of that one subpixel of type 1 is 50, then the pixel value of the subpixel of type 1 is determined to be 50. If a pixel unit 10 has two subpixels of type 2 and the pixel values ​​of those two subpixels of type 2 are 60 and 70 respectively, then the average of 60 and 70 is taken as the pixel value of the subpixel of type 2.

[0049] If there are at least two subpixels 11 of a certain type in the pixel unit 10, calculating the average value of the pixel values ​​of at least two such subpixels 11 can reduce the influence of random noise and measurement errors on the pixel value of a single subpixel 11, thereby reducing the amount of data that needs to be processed, simplifying the subsequent dye component separation calculation process, and improving algorithmic efficiency.

[0050] In some embodiments, the color filter array (CFA) of the imaging device can be a Bayer array, and the Bayer array includes four types of arrangements: RGGB, GRBG, GBRG, and BGGR. Each pixel unit 10 has multiple subpixels 11, including R pixels, G pixels, and B pixels, with one R pixel and one B pixel, and two G pixels.

[0051] When the color filter array is a Bayer array, as shown in Figure 3, each pixel unit 10 has multiple subpixels 11, including R pixels 11a, G pixels 11b, and B pixels 11c, with one R pixel 11a and one B pixel 11c per pixel unit 10, and two G pixels 11b per pixel unit 10.

[0052] In other embodiments, the color filter array of the imaging device may be of other types, such as RYYB (see Figure 4), RCCB, etc., where Y is yellow and C is transparent, such as oblique Bayer, X-Trans CMOS (Fujifilm), or quad Bayer.

[0053] In some embodiments, the various subpixels 11 in each pixel unit 10 include R pixels 11a, G pixels 11b, and B pixels 11c, with one R pixel 11a and one B pixel 11c, and two G pixels 11b. Determining the pixel values ​​of the various subpixels 11 in each pixel unit 10 includes determining that the pixel value of the R pixel 11a in each pixel unit 10 is the pixel value of one R pixel 11a, determining that the pixel value of the B pixel 11c in each pixel unit 10 is the pixel value of one B pixel 11c, and determining that the pixel value of the G pixel 11b in each pixel unit 10 is the average value of two G pixels 11b. The pixel value of the G pixel 11b in each pixel unit 10 can be the arithmetic mean or weighted mean of the pixel values ​​of two G pixels 11b.

[0054] Because the human eye is more sensitive to green light than to red and blue light, doubling the number of green pixels to twice the number of red and blue pixels allows for a better match to the visual characteristics of the human eye and achieves a more natural color reduction effect.

[0055] For example, if one pixel unit 10 has a pixel value of 80 for one R pixel 11a, a pixel value of 150 for one B pixel 11c, and two G pixels 11b have pixel values ​​of 120 and 100 respectively, then the pixel value of the R pixel 11a in the pixel unit 10 is 80, the pixel value of the B pixel 11c is 150, and the pixel value of the G pixel 11b is (120 + 100) / 2 = 110.

[0056] Figure 6 is a schematic diagram showing the distribution point D_pixel_1 of skin pixels in density space for a sample image relating to one embodiment of the present invention. In some embodiments, the color filter array of the imaging device is a Bayer array, and each pixel unit 10 includes one R pixel, one B pixel, and two G pixels. Figure 6 shows the density values ​​in density space for each sub-pixel 11 of each pixel unit 10 when the color filter array is a Bayer array. The R axis, G axis, and B axis of density space are -log10R, -log10G, and -log10B respectively, that is, the density values ​​corresponding to the R pixel, the density value corresponding to the G pixel, and the density value corresponding to the B pixel, respectively. By determining the density values ​​corresponding to each sub-pixel 11 of a given pixel unit 10, the position in density space for each sub-pixel 11 of that pixel unit 10 can be determined.

[0057] JPEG0007888809000002.jpg37170

[0058] JPEG0007888809000003.jpg47170

[0059] JPEG0007888809000004.jpg13170

[0060] JPEG0007888809000005.jpg64170

[0061] JPEG0007888809000006.jpg49170

[0062] JPEG0007888809000007.jpg14170

[0063] JPEG0007888809000008.jpg8170

[0064] In some embodiments, the process of separating pigment components based on a first pigment component distribution point D_pigment_1 for each sample image to obtain hemoglobin and melanin components includes multiplying the coordinate values ​​of the first pigment component distribution point D_pigment_1 for each sample image by a separation matrix to obtain hemoglobin and melanin components.

[0065] The segregation matrix can be obtained by independent component analysis using algorithms such as FastICA or Infomax.

[0066] By multiplying the coordinates of the first pigment component distribution point D_pigment_1 by the separation matrix, accurate separation of pigment components can be achieved.

[0067] JPEG0007888809000009.jpg79170

[0068] Figure 8 is a further subflowchart of step S16 in Figure 2. In some embodiments, determining the pigment concentration distribution determination range on the unit plane P for the measurement light source based on the hemoglobin and melanin components of each sample image in step S16, as shown in Figure 8, includes the following steps S161 and S162.

[0069] S161. Based on the hemoglobin and melanin components of each sample image, the effective range of pigment concentration distribution on the unit plane P is determined for each sample image.

[0070] S162. Based on the effective ranges of multiple dye density distributions in multiple sample images, the determination range of the dye density distribution on the unit plane P is determined for the measurement light source.

[0071] By determining the effective range of the dye concentration distribution for each sample image, outliers and noise interference can be eliminated, thereby improving the reliability of the dye concentration distribution data for each sample image.

[0072] By determining the pigment density distribution determination range based on the effective range of pigment density distribution in multiple sample images, errors and deviations due to a single sample can be reduced. The differences and diversity between different samples help to make the pigment density distribution determination range more robust, stable, and consistent. Therefore, the estimation accuracy of the target light source can be improved, and the reliability of the estimation results can be enhanced.

[0073] Figure 9 is a further subflowchart of step S161 in Figure 8. In some embodiments, determining the effective range of pigment concentration distribution on the unit plane P for each sample image based on the hemoglobin and melanin components of each sample image in step S161, as shown in Figure 9, includes the following steps S1611 to S1613.

[0074] S1611: Based on the hemoglobin components of each sample image, the mean hemoglobin concentration and the standard deviation hemoglobin concentration for each sample image are determined.

[0075] S1612: Based on the melanin components of each sample image, the average melanin concentration and the standard deviation of melanin concentration for each sample image are determined.

[0076] S1613. Based on the average hemoglobin concentration, standard deviation of hemoglobin concentration, average melanin concentration, and standard deviation of melanin concentration for each sample image, the effective range of pigment concentration distribution on the unit plane P is determined for each sample image.

[0077] By determining the effective range of the concentration distribution based on the mean and standard deviation, the reliability of the effective range of the concentration distribution can be improved.

[0078] JPEG0007888809000010.jpg61170

[0079] JPEG0007888809000011.jpg53170

[0080] JPEG0007888809000012.jpg46170

[0081] Figure 10 is a further subflowchart of step S1613 in Figure 9. In some embodiments, determining the effective range of pigment concentration distribution on the unit plane P for each sample image based on the mean hemoglobin concentration, standard deviation hemoglobin concentration, mean melanin concentration, and standard deviation melanin concentration for each sample image in step S1613, as shown in Figure 10, includes the following steps S16131 to S16133.

[0082] S16131, based on the average hemoglobin concentration and standard deviation of hemoglobin concentration for each sample image, the effective range of hemoglobin concentration distribution on the unit plane P is determined for each sample image.

[0083] S16132. Based on the average melanin concentration and standard deviation of melanin concentration for each sample image, the effective range of melanin concentration distribution on the unit plane P is determined for each sample image.

[0084] S16133, based on the effective range of hemoglobin concentration distribution and melanin concentration distribution for each sample image, the effective range of pigment concentration distribution on the unit plane P is determined for each sample image.

[0085] Since pigments mainly consist of melanin and hemoglobin, the effective range of concentration distribution for each of the two pigment components can be obtained based on the mean and standard deviation of their respective concentrations. By combining these effective ranges, the effective range of pigment concentration distribution can be further determined.

[0086] JPEG0007888809000013.jpg48170

[0087] JPEG0007888809000014.jpg44170

[0088] The confusion matrix is ​​the inverse of the separable matrix, and the confusion matrix can be determined based on the separable matrix.

[0089] JPEG0007888809000015.jpg29170

[0090] JPEG0007888809000016.jpg22170

[0091] JPEG0007888809000017.jpg55170

[0092] JPEG0007888809000018.jpg39170

[0093] JPEG0007888809000019.jpg49170

[0094] JPEG0007888809000020.jpg42170

[0095] JPEG0007888809000021.jpg13170

[0096] In some examples, the FastICA algorithm is used to separate pigment components to obtain melanin and hemoglobin components. The FastICA algorithm performs a centering process to improve efficiency and stability. After centering, the average value of the x-coordinates of all points in the first pigment component distribution point D_pigment_1 is equal to the x-coordinate of the center position C, and the average value of the y-coordinates of all points in the first pigment component distribution point D_pigment_1 is equal to the y-coordinate of the center position C.

[0097] In some embodiments, as shown in Figure 11, the hemoglobin component and the effective range of hemoglobin concentration distribution exhibit a linear distribution on the unit plane P, and the melanin component and the effective range of melanin concentration distribution also exhibit a linear distribution on the unit plane P.

[0098] Figure 12 is a further subflowchart of step S16133, and Figure 13 is a schematic diagram showing the effective range R_p of the pigment concentration distribution according to one embodiment of the present application. In some embodiments, determining the effective range of pigment concentration distribution on the unit plane P for each sample image, based on the effective range of hemoglobin concentration distribution and the effective range of melanin concentration distribution for each sample image in step S16133 as shown in Figure 12, includes the following steps S161331 and S161332.

[0099] S161331, based on the effective range R_h of hemoglobin concentration distribution and the effective range R_m of melanin concentration distribution for each sample image, the first boundary point B1, second boundary point B2, third boundary point B3, and fourth boundary point B4 located on the unit plane P are determined (see Figure 13).

[0100] S161332, the first boundary point B1, the second boundary point B2, the third boundary point B3, and the fourth boundary point B4 are connected in sequence to form the first quadrilateral Q1 which defines the effective range R_p of the dye concentration distribution (see Figure 13).

[0101] As shown in Figure 13, the connecting lines between the first boundary point B1, the first endpoint P1 of the effective melanin concentration distribution range R_m, and the second boundary point B2 are parallel to the linear distribution direction of the effective hemoglobin concentration distribution range R_h; the connecting lines between the third boundary point B3, the second endpoint P2 of the effective melanin concentration distribution range R_m, and the fourth boundary point B4 are parallel to the linear distribution direction of the effective hemoglobin concentration distribution range R_h; the connecting lines between the first boundary point B1, the third endpoint P3 of the effective hemoglobin concentration distribution range R_h, and the fourth boundary point B4 are parallel to the linear distribution direction of the effective melanin concentration distribution range R_m; and the connecting lines between the second boundary point B2, the fourth endpoint P4 of the effective hemoglobin concentration distribution range R_h, and the third boundary point B3 are parallel to the linear distribution direction of the effective melanin concentration distribution range R_m.

[0102] Figure 14 is another further subflowchart of step S1613 of Figure 9. Figure 15 is a schematic diagram showing the first boundary point B1, second boundary point B2, third boundary point B3, and fourth boundary point B4 relating to one embodiment of the present application. In some embodiments, determining the effective range of pigment concentration distribution on the unit plane P for each sample image based on the mean hemoglobin concentration, standard deviation hemoglobin concentration, mean melanin concentration, and standard deviation melanin concentration for each sample image in step S1613, as shown in Figure 14, includes the following steps S16134 to S16138.

[0103] JPEG0007888809000022.jpg16170

[0104] JPEG0007888809000023.jpg15170

[0105] JPEG0007888809000024.jpg15170

[0106] JPEG0007888809000025.jpg15170

[0107] S16138, the first boundary point B1, the second boundary point B2, the third boundary point B3, and the fourth boundary point B4 are connected in sequence to form the first quadrilateral Q1 which defines the effective range R_p of the dye concentration distribution (see Figure 15).

[0108] JPEG0007888809000026.jpg35170

[0109] Figure 16 is a further subflowchart of step S162 in Figure 8. Figure 17 is a schematic diagram showing the effective range R_p and the determination range R_d of the dye density distribution of multiple sample images relating to one embodiment of the present application. As shown in Figure 16, determining the determination range of the dye density distribution on the unit plane P for the measurement light source based on the multiple effective ranges of the dye density distribution of multiple sample images in S162 includes the following steps S1621 to S1625.

[0110] S1621. Based on the position of the first boundary point B1 of the effective range R_p of the pigment density distribution of all sample images, the position of the fifth boundary point B5 of the pigment density distribution determination range R_d is determined (see Figure 17).

[0111] S1622, based on the position of the second boundary point B2 of the effective range R_p of the pigment density distribution of all sample images, the position of the sixth boundary point B6 of the determination range R_d of the pigment density distribution is determined (see Figure 17).

[0112] S1623, based on the position of the third boundary point B3 of the effective range R_p of the pigment density distribution of all sample images, the position of the seventh boundary point B7 of the determination range R_d of the pigment density distribution is determined (see Figure 17).

[0113] S1624, based on the position of the fourth boundary point B4 of the effective range R_p of the pigment density distribution of all sample images, the position of the eighth boundary point B8 of the determination range R_d of the pigment density distribution is determined (see Figure 17).

[0114] S1625, the fifth boundary point B5, the sixth boundary point B6, the seventh boundary point B7, and the eighth boundary point B8 are connected in sequence to form the second quadrilateral Q2 which defines the pigment concentration distribution determination range R_d (see Figure 17).

[0115] By determining the positions of the 5th boundary point B5, 6th boundary point B6, 7th boundary point B7, and 8th boundary point B8 based on the positions of the boundary points of the effective range R_p of the dye density distribution corresponding to multiple sample images, errors and deviations due to a single sample can be reduced. Therefore, the positional accuracy of the 5th boundary point B5, 6th boundary point B6, 7th boundary point B7, and 8th boundary point B8 can be improved, and furthermore, the accuracy of the dye density distribution determination range R_d can be improved, thereby improving the estimation accuracy of the target light source.

[0116] Determining the position of the fifth boundary point B5 of the pigment density distribution determination range R_d based on the position of the first boundary point B1 of the pigment density distribution effective range R_p of all sample images includes taking the average of the horizontal and vertical coordinate values ​​of the first boundary point B1 of the pigment density distribution effective range R_p of all sample images as the horizontal and vertical coordinate values ​​of the fifth boundary point B5, respectively. The position of the fifth boundary point B5 refers to the position of the fifth boundary point B5 on the unit plane P.

[0117] Determining the position of the sixth boundary point B6 of the pigment density distribution determination range R_d based on the position of the second boundary point B2 of the pigment density distribution effective range R_p of all sample images involves using the average of the horizontal and vertical coordinate values ​​of the second boundary point B2 of the pigment density distribution effective range R_p of all sample images as the horizontal and vertical coordinate values ​​of the sixth boundary point B6, respectively. The position of the sixth boundary point B6 refers to the position of the sixth boundary point B6 on the unit plane P.

[0118] Determining the position of the seventh boundary point B7 of the pigment density distribution determination range R_d based on the position of the third boundary point B3 of the pigment density distribution effective range R_p of all sample images involves using the average of the horizontal and vertical coordinate values ​​of the third boundary point B3 of the pigment density distribution effective range R_p of all sample images as the horizontal and vertical coordinate values ​​of the seventh boundary point B7, respectively. The position of the seventh boundary point B7 refers to the position of the seventh boundary point B7 on the unit plane P.

[0119] Determining the position of the eighth boundary point B8 of the pigment density distribution determination range R_d based on the position of the fourth boundary point B4 of the pigment density distribution effective range R_p of all sample images involves using the average horizontal and vertical values ​​of the fourth boundary point B4 of the pigment density distribution effective range R_p of all sample images as the horizontal and vertical values ​​of the eighth boundary point B8, respectively. The position of the eighth boundary point B8 refers to the position of the eighth boundary point B8 on the unit plane P.

[0120] JPEG0007888809000027.jpg102170

[0121] JPEG0007888809000028.jpg17170

[0122] Figure 18 is a schematic diagram showing multiple dye concentration distribution determination ranges R_d and multiple dye concentration distribution effective ranges R_p corresponding to each measurement light source in one embodiment of the present application. In some embodiments, the multiple measurement light sources are D65, U30, TL84, CWF, A, and HZ, respectively. Figure 18 shows multiple effective pigment concentration distribution ranges R_p_D65 and one pigment concentration distribution determination range R_d_D65 corresponding to D65, multiple effective pigment concentration distribution ranges R_p_U30 and one pigment concentration distribution determination range R_d_U30 corresponding to U30, multiple effective pigment concentration distribution ranges R_p_TL84 and one pigment concentration distribution determination range R_d_TL84 corresponding to TL84, multiple effective pigment concentration distribution ranges R_p_CWF and one pigment concentration distribution determination range R_d_CWF corresponding to CWF, multiple effective pigment concentration distribution ranges R_p_A and one pigment concentration distribution determination range R_d_A corresponding to A, and multiple effective pigment concentration distribution ranges R_p_HZ and one pigment concentration distribution determination range R_d_HZ corresponding to HZ.

[0123] In some embodiments, determining the type of estimated target light source based on the proportion of pixels located within each pigment density distribution determination range R_d for a target image includes determining that the type of estimated target light source is the same as the type of measurement light source corresponding to the pigment density distribution determination range R_d with the highest proportion.

[0124] In other words, if the proportion of pixels located within a certain pigment density distribution determination range R_d in the target image is the highest, then it can be determined that the type of measurement light source corresponding to that pigment density distribution determination range R_d and the type of estimated target light source are the same. When capturing images with the same imaging device, if different objects are captured under the same or similar type of light source, the pigment distribution regions of the obtained images will be similar. Therefore, by using the type of measurement light source corresponding to the pigment density distribution determination range R_d with the highest proportion as the estimated target light source type, highly accurate light source estimation can be achieved.

[0125] The proportion of pixels located within the pigment density distribution determination range R_d for each target image refers to the ratio of the number of pigment component distribution points in the target image located within the pigment density distribution determination range R_d to the total number of pigment component distribution points in the target image.

[0126] Figure 19 is a schematic diagram showing multiple pigment density distribution determination ranges R_d corresponding to multiple measurement light sources and the second pigment component distribution point D_pigment_2 of the target image, relating to one embodiment of the present application. The pigment component distribution point of the target image is indicated by the second pigment component distribution point D_pigment_2. As shown in Figure 19, the pigment density distribution determination range R_d with the highest proportion of pixels located within the pigment density distribution determination range R_d for the second pigment component distribution point D_pigment_2 of the target image corresponds to measurement light source A, and the type of the estimated target light source is the same as the type of measurement light source A, i.e., the estimated target light source is light source A.

[0127] In some embodiments, determining the pigment density distribution determination range R_d on the unit plane P for a measurement light source based on multiple effective pigment density distribution ranges R_p of multiple sample images includes defining the pigment density distribution determination range R_d as the superimposed region of the effective pigment density distribution ranges R_p of all sample images.

[0128] In some embodiments, defining the overlapping region of the effective pigment density distribution range R_p of all sample images as the pigment density distribution determination range R_d means defining the overlapping region of the first quadrilateral Q1 corresponding to all sample images as the pigment density distribution determination range R_d.

[0129] The superimposed region reflects the common characteristics of the effective pigment density distribution range R_p of multiple sample images, enhancing the versatility of the pigment density distribution determination range R_d, and allowing it to be applied to different subjects and shooting scenes.

[0130] Figure 20 is a further subflowchart of step S7 in Figure 1. In some embodiments, determining the type of estimated target light source based on the percentage of pixels located within each pigment density distribution determination range for the target image in step S7, as shown in Figure 20, includes the following steps S71 and S72.

[0131] S71. Based on the proportion of pixels located within each pigment density distribution determination range R_d for the target image, the corresponding weights for each pigment density distribution determination range R_d are determined.

[0132] S72. Based on the weights corresponding to multiple pigment concentration distribution determination ranges R_d, the light source color of the estimated target light source is determined by applying a weighted average to the light source colors of the measurement light sources corresponding to multiple pigment concentration distribution determination ranges R_d.

[0133] For the target image, the proportion of pixels located within each pigment density distribution determination range R_d is the ratio of the number of points located within the pigment density distribution determination range R_d for each second pigment component distribution point D_pigment_2 of the target image to the total number of second pigment component distribution points D_pigment_2 of the target image.

[0134] In some cases, particularly when the positions of the pigment density distribution determination range R_d of multiple sample images are close together, the second pigment component distribution point D_pigment_2 of the target image may be located within multiple pigment density distribution determination ranges R_d. By performing a weighted average of the light source colors of the measurement light sources corresponding to the multiple pigment density distribution determination ranges R_d based on the proportion of pixels located within each pigment density distribution determination range R_d of the target image, the calculated estimated light source color can be made more accurate, and the estimation accuracy of the estimated target light source can be further improved.

[0135] The higher the proportion of pixels in a target image that fall within a certain pigment density distribution determination range R_d, the larger the weight corresponding to that pigment density distribution determination range R_d.

[0136] Figure 21 is a schematic diagram showing multiple pigment density distribution determination ranges R_d corresponding to multiple measurement light sources and the second pigment component distribution point D_pigment_2 of the target image, relating to other embodiments of the present invention. As shown in Figure 21, a portion of the second pigment component distribution point D_pigment_2 of the target image is located within the pigment density distribution determination range R_d corresponding to the measurement light sources TL84, CWF, and D65, and the proportion of pixels in the second pigment component distribution point D_pigment_2 of the target image that are located within the pigment density distribution determination range R_d corresponding to the measurement light sources TL84, CWF, and D65 gradually decreases. The weights corresponding to the pigment density distribution determination range R_d of the measurement light sources TL84, CWF, and D65 gradually decrease.

[0137] JPEG0007888809000029.jpg41170

[0138] JPEG0007888809000030.jpg53170

[0139] JPEG0007888809000031.jpg58170

[0140] Referring to Figures 22 and 23, Figure 22 is another further subflowchart of step S1613 of Figure 9, and Figure 23 is a schematic diagram showing the effective range R_p of the pigment concentration distribution relating to another embodiment of the present application. As shown in Figure 22, determining the effective range of the pigment concentration distribution on the unit plane P for each sample image based on the mean hemoglobin concentration, standard deviation hemoglobin concentration, mean melanin concentration, and standard deviation melanin concentration for each sample image in step S1613 includes the following steps S221 to S223.

[0141] In S221, "c5, c6" and "c7, c8" are multiplied by the mixing matrix, respectively, to obtain the coordinates of the first endpoint P1 and the second endpoint P2 of the effective range R_m of the melanin concentration distribution on the unit plane P (see Figure 23).

[0142] In S222, "c1, c2" and "c3, c4" are multiplied by the mixing matrix, respectively, to obtain the coordinates of the third endpoint P3 and the fourth endpoint P4 of the effective range R_h of the hemoglobin concentration distribution on the unit plane P (see Figure 23).

[0143] In S223, the first and second endpoints P1 and P2 are defined as the two endpoints of the first axis, and the third and fourth endpoints P3 and P4 are defined as the two endpoints of the second axis, thereby obtaining the first oblique ellipse E1 that defines the effective range R_p of the pigment concentration distribution (see Figure 23).

[0144] The first axis is one of the major and minor axes of the first oblique ellipse E1, and the second axis is the other of the major and minor axes of the first oblique ellipse E1. The angle between the major and minor axes of the first oblique ellipse E1 is not a right angle.

[0145] In some embodiments, determining the pigment density distribution determination range R_d on the unit plane P for a measurement light source based on multiple effective pigment density distribution ranges R_p of multiple sample images includes determining a second oblique ellipse that defines the pigment density distribution determination range R_d on the unit plane P for the measurement light source based on a first oblique ellipse E1 corresponding to multiple sample images.

[0146] Comparing obtaining the pigment concentration distribution determination range R_d based on the first oblique ellipse E1 with obtaining it based on the first quadrilateral Q1 or the second quadrilateral Q2, the accuracy of the pigment concentration distribution determination range R_d obtained based on the first quadrilateral Q1 or the second quadrilateral Q2 is slightly lower, but the computational load is reduced, allowing for faster acquisition of the pigment concentration distribution determination range R_d.

[0147] Referring to Figures 24 to 29, Figure 24 is a further subflowchart of step S6 of Figure 1, Figure 25 is a schematic diagram showing the pigment concentration distribution determination range R_d, hemoglobin concentration distribution effective range R_h, melanin concentration distribution effective range R_m, and second pigment component distribution point D_pigment_2 corresponding to a sample image on the unit plane P before step S61 is performed, relating to one embodiment of the present application, Figure 26 is a schematic diagram showing the pigment concentration distribution determination range R_d, hemoglobin concentration distribution effective range R_h, melanin concentration distribution effective range R_m, and second pigment component distribution point D_pigment_2 corresponding to a sample image on the unit plane P after step S61 and before step S62 is performed, relating to one embodiment of the present application, and Figure 28 is a schematic diagram showing the pigment concentration distribution determination range R_d, hemoglobin concentration distribution effective range R_h, melanin concentration distribution effective range R_m, and second pigment component distribution point D_pigment_2 corresponding to the sample image on the unit plane P before step S63 is performed. Figure 28 is a schematic diagram showing the pigment concentration distribution determination range R_d, hemoglobin concentration distribution effective range R_h, melanin concentration distribution effective range R_m, and second pigment component distribution point D_pigment_2 corresponding to the sample image on the unit plane P after step S63 is performed and before step S64 is performed, according to one embodiment of the present application. Figure 29 is a schematic diagram showing the updated pigment concentration distribution determination range R_d' and updated second pigment component distribution point D_pigment_2' on the unit plane P after step S64 is performed, according to one embodiment of the present application. In some embodiments, as shown in Figure 24, determining the proportion of pixels located within each pigment density distribution determination range for the target image based on the pigment component distribution points of the target image and the corresponding pigment density distribution determination range for each measurement light source in step S6, that is, determining the proportion of pixels located within each pigment density distribution determination range R_d for the target image based on the second pigment component distribution point D_pigment_2 and the corresponding pigment density distribution determination range R_d for each measurement light source, includes the following steps S61 to S65.

[0148] S61, the pigment density distribution determination range R_d corresponding to the sample image and the second pigment component distribution point D_pigment_2 of the target image are simultaneously translated within the unit plane P, and the translation continues until the center of the pigment density distribution determination range R_d coincides with the origin O. For example, as shown in Figure 25, the pigment density distribution determination range R_d and the second pigment component distribution point D_pigment_2 corresponding to the sample image are translated along the first arrow A1, and the translation continues until the center of the pigment density distribution determination range R_d coincides with the origin O (see Figure 26). The center of the pigment density distribution determination range R_d can be understood as the center of the second oblique ellipse E2, that is, the intersection of the first and second axes mentioned above.

[0149] In S62, the first effective concentration distribution range is rotated around the origin O until it coincides with the first target coordinate axis. The first effective concentration distribution range is either the hemoglobin concentration distribution range R_h or the melanin concentration distribution range R_m corresponding to each sample image, and the first target coordinate axis is either the first coordinate axis a or the second coordinate axis b. For example, as shown in Figure 26, the first effective concentration distribution range is the melanin concentration distribution range R_m corresponding to each sample image, and the first target coordinate axis is the second coordinate axis b. The melanin concentration distribution range R_m is rotated around the origin O along the second arrow A2 until it coincides with the second coordinate axis b (see Figure 27).

[0150] In S63, the pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 are sheared along a direction parallel to the first target coordinate axis until the effective range of the second concentration distribution overlaps with the second target coordinate axis. The effective range of the second concentration distribution is the other of the effective hemoglobin concentration distribution range R_h and the effective melanin concentration distribution range R_m corresponding to each sample image, and the second target coordinate axis is the other of the first coordinate axis a and the second coordinate axis b. For example, as shown in Figure 27, the first target coordinate axis is the second coordinate axis b, the effective range of the second concentration distribution is the effective hemoglobin concentration distribution range R_h corresponding to each sample image, and the second target coordinate axis is the first coordinate axis a. The pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 are sheared along a direction parallel to the second coordinate axis b so that the effective hemoglobin concentration distribution range R_h overlaps with the first coordinate axis a (see Figure 28). Along the third arrow A3, the portion located above the second coordinate axis b in the pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 is sheared; that is, along the third arrow A3, the portion of the pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 where the a-coordinate is positive is sheared. Along the fourth arrow A4, the portion located below the second coordinate axis b in the pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 is sheared; that is, along the fourth arrow A4, the portion of the pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 where the a-coordinate is negative is sheared.

[0151] In S64, the pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 are compressed along a direction parallel to the first target coordinate axis to obtain an updated pigment concentration distribution determination range and an updated second pigment component distribution point, and the outline of the updated pigment concentration distribution determination range R_d' constitutes the first perfect circle. For example, as shown in Figure 28, the first target coordinate axis is the second coordinate axis b, and the pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 are compressed along a direction parallel to the second coordinate axis b to obtain an updated pigment concentration distribution determination range R_d' and an updated second pigment component distribution point D_pigment_2', and the outline of the updated pigment concentration distribution determination range R_d' constitutes the first perfect circle C1 (see Figure 29). Along the fifth arrow A5, the portion located to the right of the first coordinate axis a in the pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 is compressed; that is, along the fifth arrow A5, the portion of the pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 where the b coordinate is positive is compressed. Along the sixth arrow A6, the portion located to the left of the first coordinate axis a in the pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 is compressed; that is, along the sixth arrow A6, the portion of the pigment concentration distribution determination range R_d and the second pigment component distribution point D_pigment_2 where the b coordinate is negative is compressed.

[0152] In S65, the proportion of pixels located within the updated pigment concentration distribution determination range R_d' at the updated second pigment component distribution point D_pigment_2' is determined.

[0153] The estimated target light source type is the same as the measurement light source type corresponding to the updated dye concentration distribution determination range R_d' with the highest proportion.

[0154] In some embodiments, the first target coordinate axis is the first coordinate axis a, and the second target coordinate axis is the second coordinate axis b. In other embodiments, the first target coordinate axis is the second coordinate axis b, and the second target coordinate axis is the first coordinate axis a.

[0155] In some embodiments, the first effective concentration distribution range is the melanin concentration distribution range R_m, and the second effective concentration distribution range is the hemoglobin concentration distribution range R_h. In other embodiments, the first effective concentration distribution range is the hemoglobin concentration distribution range R_h, and the second effective concentration distribution range is the melanin concentration distribution range R_m.

[0156] In step S65, the distance between each point of the updated second pigment component distribution point D_pigment_2' and the origin O, and the radius of the first circle C1 are calculated. The distance and radius are compared, and based on the comparison result, it can be determined whether or not the point is located within the updated pigment concentration distribution determination range R_d'. For example, as shown in Figure 29, the distance between the first point D1 and the origin O is S1, the distance between the second point D2 and the origin O is S2, and the radius of the first circle C1 is r. Since S1 is greater than r and S2 is less than r, the first point D1 is located outside the updated pigment concentration distribution determination range R_d', and the second point D2 is located within the updated pigment concentration distribution determination range R_d'. Furthermore, it can be determined whether or not all points of the updated second pigment component distribution point D_pigment_2' are located within the updated pigment concentration distribution determination range R_d'. Therefore, it is possible to obtain the proportion of pixels located within the updated pigment concentration distribution determination range R_d' at the updated second pigment component distribution point D_pigment_2'.

[0157] In S62 to S65, the pigment concentration distribution determination range R_d may be defined by the second quadrilateral Q2 or by the second oblique ellipse E2.

[0158] The embodiments of the present invention further provide an image processing method.

[0159] In some embodiments, the image processing method includes determining the type of light source to be estimated by the light source estimation method based on the distribution of skin pigment components described in any one of the embodiments described above; determining a target white balance gain and / or target color correction value based on the determined type of light source to be estimated; and processing an image acquired under the light source to be estimated based on the target white balance gain and / or target color correction value. The image in "processing an image acquired under the light source to be estimated" may be an image of the subject or an image of another subject when estimating the light source to be estimated.

[0160] The light source estimation method provided by the embodiment of the present invention can accurately determine the type of light source to be estimated, and furthermore, can improve the image processing quality by processing an image acquired under the estimated light source using the white balance gain and / or color correction value corresponding to the estimated light source.

[0161] Determining the target white balance gain and / or target color correction value based on the determined estimated target light source type includes a preset correspondence between the light source type and the white balance gain and / or color correction value, and determining the target white balance gain and / or target color correction value based on the determined estimated target light source type.

[0162] By pre-calculating the white balance gain and / or color correction value corresponding to each light source type, a pre-defined correspondence between the light source type and the white balance gain and / or color correction value can be obtained. The pre-defined correspondence between the light source type and the white balance gain and / or color correction value defines that there is one correspondence between multiple light source types and multiple white balance gains and / or color correction values, meaning that each light source type corresponds to one white balance gain and / or one color correction value.

[0163] The embodiments of the present application further provide an imaging device.

[0164] Figure 30 is a schematic diagram showing the structure of an imaging device 100 according to one embodiment of the present application. As shown in Figure 30, the imaging device 100 includes an image acquisition unit 1, a determination range calculation unit 2, and a light source estimation unit 3.

[0165] The image acquisition unit 1 acquires sample images corresponding to multiple samples separately under each measurement light source and is used to determine the skin area for each sample image. The image acquisition unit 1 further acquires a target image of the subject under the estimated target light source and is used to determine the skin area of ​​the target image.

[0166] The determination range calculation unit 2 is used to determine the pigment concentration distribution determination range R_d for multiple measurement light sources of different types, separately for each measurement light source, in order to obtain multiple pigment concentration distribution determination ranges R_d corresponding to the multiple measurement light sources, with each measurement light source corresponding to one pigment concentration distribution determination range R_d. The determination range calculation unit 2 can perform the above-described step S1.

[0167] JPEG0007888809000032.jpg45170

[0168] JPEG0007888809000033.jpg61170

[0169] The determination range calculation unit 2 can execute the steps S11 to S16 and the subdivision steps of steps S11 to S16 described above. For example, it can execute the subdivision steps S131 to S132 of step S13, the subdivision steps S161 to S162 of step S16, the subdivision steps S1611 to S1613 of step S1613 in one embodiment, the subdivision steps S161331 to S161332 of step S16133, the subdivision steps S16134 to S16138 of step S1613 in another embodiment, and the subdivision steps S1621 to S1625 of step S162.

[0170] The determination range calculation unit 2 can execute the subdivision steps S4 to S7. For example, it can execute the subdivision steps S71 to S72 of step S7.

[0171] Figure 31 is a schematic diagram showing the structure of an imaging device 100 according to another embodiment of the present invention. In some embodiments, as shown in Figure 31, the imaging device 100 includes an image acquisition unit 1, a determination range calculation unit 2, a light source estimation unit 3, and a color correction unit 4. The color correction unit 4 determines a target white balance gain and / or target color correction value based on the type of estimated target light source determined by the light source estimation unit 3, and is used to process an image acquired under the estimated target light source based on the target white balance gain and / or target color correction value.

[0172] The image acquisition unit 1 may include an image sensor unit, an optical lens unit, a lens drive unit, etc. The image sensor unit is used to convert optical signals into image signals and generate raw data for each pixel corresponding to the optical image of the sample or subject. The optical lens unit includes a focusing lens for focusing light from the sample or subject onto the image sensor unit. The lens drive unit is used to drive the focusing lens in order to adjust its position.

[0173] The determination range calculation unit 2 and the light source estimation unit 3 may include processing chips such as microcontrollers, CPUs (central processing units), and DSPs (digital signal processors), or hardware elements in processing chips such as microcontrollers, CPUs, and DSPs, or software program modules written to processing chips such as microcontrollers, CPUs, and DSPs.

[0174] The functions performed by the imaging device 100 correspond to the light source estimation method and image processing method described above. For a more detailed explanation, refer to the contents of each embodiment of the light source estimation method and image processing method. The contents of the imaging device 100 and the light source estimation method and image processing method described above can also be referenced from each other.

[0175] The configurations illustrated in the embodiments of this application do not specifically limit the imaging device 100. In other embodiments, the imaging device 100 may include more or fewer components than those shown, or may have combinations of some components, or divisions of some components, or different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0176] The present invention further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is loaded into a processor and executed, the light source estimation method and / or the image processing method described in the above-described arbitrary embodiment are realized.

[0177] Computer-readable storage media include various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] The foregoing describes only specific embodiments of the present application, and the scope of protection of the present application is not limited thereto. Those skilled in the art can easily conceive of any modifications or substitutions within the scope of the art disclosed herein, and all such modifications or substitutions should be included within the scope of this application. Embodiments and features of embodiments of the present application may be combined, provided they do not conflict. Therefore, the scope of protection of the present application should be determined by the claims. [Explanation of symbols]

[0179] 10…Pixel unit, 11…Subpixel, 11a…R pixel, 11b…G pixel, 11c…B pixel, P…Unit plane, D_pixel_1…Skin pixel distribution point, D_pigment_1…First pigment component distribution point, P1…First endpoint, P2…Second endpoint, P3…Third endpoint, P4…Fourth endpoint, R_h…Effective range of hemoglobin concentration distribution, R_p…Effective range of pigment concentration distribution, R_m…Effective range of melanin concentration distribution, B1…First boundary point, B2…Second boundary point, B3…Third boundary point, B4…Fourth boundary point B5...5th boundary point, B6...6th boundary point, B7...7th boundary point, B8...8th boundary point, D_pigment_2...2nd pigment component distribution point, Q1...1st quadrilateral, R_d...pigment density distribution determination range, Q2...2nd quadrilateral, E1...1st oblique ellipse, E2...2nd oblique ellipse, R_d'...updated pigment density distribution determination range, D_pigment_2'...updated 2nd pigment component distribution point, C1...1st perfect circle, 100...imaging device, 1...image acquisition unit, 2...determination range calculation unit, 3...light source estimation unit, 4...color correction unit.

Claims

1. A method for estimating the light source based on the distribution of skin pigment components, The determination range calculation unit determines the pigment concentration distribution determination range for each of the multiple measurement light sources, each of which is of a different type, separately, and obtains multiple pigment concentration distribution determination ranges corresponding to the multiple measurement light sources, wherein each measurement light source corresponds to one pigment concentration distribution determination range. The image acquisition unit obtains a target image of the subject under the estimated target light source, The image acquisition unit determines the skin region of the target image, The light source estimation unit converts the pixel values ​​of the skin region in the target image into density values ​​to obtain skin pixel distribution points in density space for the target image, The light source estimation unit projects the skin pixel distribution points of the target image onto a unit plane along the direction of the shading component vector to obtain the pigment component distribution points on the unit plane for the target image. The light source estimation unit determines the proportion of pixels located within each pigment density distribution determination range for the target image, based on the pigment component distribution points of the target image and the pigment density distribution determination range corresponding to each measurement light source. The light source estimation unit determines the type of the estimated target light source based on the proportion of pixels located within each pigment density distribution determination range for the target image, including, A method for estimating a light source based on the distribution of skin pigment components, characterized by the following:

2. Determining the range for determining the pigment concentration distribution is The image acquisition unit obtains multiple sample images corresponding to multiple samples under the same measurement light source, The image acquisition unit determines the skin area for each sample image, The judgment range calculation unit converts the pixel values ​​of the skin region for each sample image into density values ​​to obtain the skin pixel distribution points in density space for each sample image, The determination range calculation unit projects the skin pixel distribution points for each sample image onto the unit plane along the direction of the shading component vector to obtain the pigment component distribution points on the unit plane for each sample image. The judgment range calculation unit performs a separation process of pigment components based on the pigment component distribution points for each sample image to obtain hemoglobin and melanin components. The determination range calculation unit determines the pigment concentration distribution determination range on the unit plane for the measurement light source based on the hemoglobin component and the melanin component for each sample image, including, The method for estimating a light source based on the distribution of skin pigment components as described in feature 1.

3. The sample image includes a plurality of pixel units, each pixel unit includes a plurality of subpixels, and the pixel values ​​of the skin region in the sample image are the pixel values ​​of the subpixels. The judgment range calculation unit converts the pixel values ​​of the skin region in each sample image into density values. The judgment range calculation unit obtains the pixel values ​​of all subpixels in the skin region for each sample image, The determination range calculation unit determines the pixel values ​​of various subpixels for each pixel unit, calculates the logarithm of the reciprocal of the pixel values ​​of various subpixels for each pixel unit, and obtains the density value corresponding to the pixel values ​​of various subpixels for each pixel unit. including, The method for estimating a light source based on the distribution of skin pigment components as described in feature 2.

4. The determination range calculation unit determines the pixel values ​​of various subpixels for each pixel unit. The determination range calculation unit determines that if the number of any one subpixel of any one type in any one pixel unit is one, the pixel value of that subpixel is determined to be the pixel value of one of that subpixel. The determination range calculation unit, if any one pixel unit has at least two subpixels of any one type, takes the average value of the pixel values ​​of at least two such subpixels as the pixel value of that subpixel. including, The method for estimating a light source based on the distribution of skin pigment components as described in feature 3.

5. Each pixel unit contains various subpixels, including R pixels, G pixels, and B pixels. Each pixel unit has one R pixel and one B pixel, and two G pixels. The determination range calculation unit determines the pixel values ​​of various subpixels for each pixel unit. The determination range calculation unit determines that the pixel value of the R pixel for each pixel unit is the pixel value of one of the R pixels, The determination range calculation unit determines that the pixel value of each B pixel in each pixel unit is the pixel value of one B pixel, The determination range calculation unit determines that the pixel value of the G pixel for each pixel unit is the average value of the two aforementioned G pixels, including, The method for estimating a light source based on the distribution of skin pigment components as described in feature 3.

6. The aforementioned shading component vector is (-1, -1, -1). The method for estimating a light source based on the distribution of skin pigment components as described in feature 1.

7. The determination range calculation unit performs a separation process of pigment components based on the pigment component distribution points for each sample image to obtain hemoglobin components and melanin components. The determination range calculation unit includes multiplying the coordinate values ​​of the pigment component distribution points for each sample image by a separation matrix to obtain the hemoglobin component and the melanin component. The method for estimating a light source based on the distribution of skin pigment components as described in feature 2.

8. The determination range calculation unit determines the pigment concentration distribution determination range on the unit plane for the measurement light source based on the hemoglobin component and the melanin component for each sample image, The determination range calculation unit determines the effective range of pigment concentration distribution on the unit plane for each sample image based on the hemoglobin component and the melanin component for each sample image, The determination range calculation unit determines the determination range of the pigment concentration distribution on the unit plane for the measurement light source based on the effective ranges of multiple pigment concentration distributions of multiple sample images. including, The method for estimating a light source based on the distribution of skin pigment components as described in feature 2.

9. The determination range calculation unit determines the effective range of pigment concentration distribution on the unit plane for each sample image based on the hemoglobin component and the melanin component for each sample image, The determination range calculation unit determines the average hemoglobin concentration and the standard deviation of hemoglobin concentration for each sample image based on the hemoglobin component of each sample image. The determination range calculation unit determines the average melanin concentration and the standard deviation of melanin concentration for each sample image based on the melanin components for each sample image. The determination range calculation unit determines the effective range of pigment concentration distribution on the unit plane for each sample image based on the average hemoglobin concentration, standard deviation of hemoglobin concentration, average melanin concentration, and standard deviation of melanin concentration for each sample image. including, The method for estimating a light source based on the distribution of skin pigment components as described in feature 8.

10. The determination range calculation unit determines the effective range of pigment concentration distribution on the unit plane for each sample image based on the average hemoglobin concentration, standard deviation of hemoglobin concentration, average melanin concentration, and standard deviation of melanin concentration for each sample image. The determination range calculation unit determines the effective range of hemoglobin concentration distribution on the unit plane for each sample image based on the average hemoglobin concentration and the standard deviation of hemoglobin concentration for each sample image, The determination range calculation unit determines the effective range of melanin concentration distribution on the unit plane for each sample image based on the average melanin concentration and standard deviation of melanin concentration for each sample image. The determination range calculation unit determines the effective range of pigment concentration distribution on the unit plane for each sample image based on the effective range of hemoglobin concentration distribution and the effective range of melanin concentration distribution for each sample image, including, The method for estimating a light source based on the distribution of skin pigment components as described in feature 9.

11. The effective range of the hemoglobin concentration distribution exhibits a linear distribution on the unit plane, and the effective range of the melanin concentration distribution exhibits a linear distribution on the unit plane. The determination range calculation unit determines the effective range of pigment concentration distribution on the unit plane for each sample image based on the effective range of hemoglobin concentration distribution and the effective range of melanin concentration distribution for each sample image. The determination range calculation unit determines the first boundary point, second boundary point, third boundary point, and fourth boundary point located on the unit plane, The determination range calculation unit sequentially connects the first boundary point, the second boundary point, the third boundary point, and the fourth boundary point to form a first quadrilateral that defines the effective range of the dye concentration distribution, Includes, The connecting line between the first boundary point, the first endpoint of the effective range of melanin concentration distribution, and the second boundary point is parallel to the linear distribution direction of the effective range of hemoglobin concentration distribution. The connecting line between the third boundary point, the second endpoint of the effective range of melanin concentration distribution, and the fourth boundary point is parallel to the linear distribution direction of the effective range of hemoglobin concentration distribution. The connecting line between the first boundary point, the third endpoint of the effective hemoglobin concentration distribution range, and the fourth boundary point is parallel to the linear distribution direction of the effective melanin concentration distribution range. The connecting line between the second boundary point, the fourth endpoint of the effective hemoglobin concentration distribution range, and the third boundary point is parallel to the linear distribution direction of the effective melanin concentration distribution range. The method for estimating a light source based on the distribution of skin pigment components according to feature 10.

12. The determination range calculation unit determines the effective range of hemoglobin concentration distribution on the unit plane for each sample image based on the average hemoglobin concentration and standard deviation of hemoglobin concentration for each sample image, which includes the determination range calculation unit multiplying "c1, c2" and "c3, c4" respectively by the mixing matrix to obtain the effective range of hemoglobin concentration distribution on the unit plane for each sample image. The determination range calculation unit determines the effective range of melanin concentration distribution on the unit plane for each sample image based on the average melanin concentration and standard deviation of melanin concentration for each sample image, which includes the determination range calculation unit multiplying "c5, c6" and "c7, c8" respectively with the mixing matrix to obtain the effective range of melanin concentration distribution on the unit plane for each sample image. The method for estimating a light source based on the distribution of skin pigment components according to feature 10.

13. The determination range calculation unit determines the effective range of pigment concentration distribution on the unit plane for each sample image based on the average hemoglobin concentration, standard deviation of hemoglobin concentration, average melanin concentration, and standard deviation of melanin concentration for each sample image. The determination range calculation unit sequentially connects the first boundary point, the second boundary point, the third boundary point, and the fourth boundary point to form a first quadrilateral that defines the effective range of the dye concentration distribution, Includes, The method for estimating a light source based on the distribution of skin pigment components as described in feature 9.

14. The determination range calculation unit determines the determination range of the pigment concentration distribution on the unit plane for the measurement light source based on the effective ranges of multiple pigment concentration distributions of multiple sample images, The determination range calculation unit determines the position of the fifth boundary point of the pigment density distribution determination range based on the position of the first boundary point of the effective pigment density distribution range of all sample images, The determination range calculation unit determines the position of the sixth boundary point of the pigment density distribution determination range based on the position of the second boundary point of the effective pigment density distribution range of all sample images. The determination range calculation unit determines the position of the seventh boundary point of the pigment density distribution determination range based on the position of the third boundary point of the effective pigment density distribution range of all sample images. The determination range calculation unit determines the position of the eighth boundary point of the pigment density distribution determination range based on the position of the fourth boundary point of the effective pigment density distribution range of all sample images, The determination range calculation unit sequentially connects the fifth boundary point, the sixth boundary point, the seventh boundary point, and the eighth boundary point to form a second quadrilateral that defines the pigment concentration distribution determination range, including, The method for estimating a light source based on the distribution of skin pigment components according to feature 11.

15. The determination range calculation unit determines the position of the fifth boundary point of the pigment density distribution determination range based on the position of the first boundary point of the effective pigment density distribution range of all sample images, which includes the determination range calculation unit setting the average value of the horizontal coordinate and the average value of the vertical coordinate of the first boundary point of the effective pigment density distribution range of all sample images as the horizontal coordinate and vertical coordinate of the fifth boundary point, respectively. The determination range calculation unit determines the position of the sixth boundary point of the pigment density distribution determination range based on the position of the second boundary point of the effective pigment density distribution range of all sample images, which includes the determination range calculation unit using the average value of the horizontal coordinate and the average value of the vertical coordinate of the second boundary point of the effective pigment density distribution range of all sample images as the horizontal and vertical coordinate values ​​of the sixth boundary point, respectively. The determination range calculation unit determines the position of the seventh boundary point of the pigment density distribution determination range based on the position of the third boundary point of the effective pigment density distribution range of all sample images, which includes the determination range calculation unit using the average value of the horizontal coordinate and the average value of the vertical coordinate of the third boundary point of the effective pigment density distribution range of all sample images as the horizontal and vertical coordinate values ​​of the seventh boundary point, respectively. The determination of the position of the eighth boundary point of the pigment density distribution determination range by the determination range calculation unit, based on the position of the fourth boundary point of the effective pigment density distribution range of all sample images, includes the determination range calculation unit using the average value of the horizontal coordinate and the average value of the vertical coordinate of the fourth boundary point of the effective pigment density distribution range of all sample images as the horizontal and vertical coordinate values ​​of the eighth boundary point, respectively. The method for estimating a light source based on the distribution of skin pigment components as described in feature 14.

16. The determination range calculation unit determines the determination range of the pigment concentration distribution on the unit plane for the measurement light source based on the effective ranges of multiple pigment concentration distributions of multiple sample images, The determination range calculation unit includes setting the overlapping region of the effective range of the pigment density distribution of all sample images as the pigment density distribution determination range. The method for estimating a light source based on the distribution of skin pigment components as described in feature 8.

17. The light source estimation unit determines the type of the estimated target light source based on the proportion of pixels located within each pigment density distribution determination range for the target image, The light source estimation unit includes determining that the type of the light source to be estimated is the same as the type of measurement light source corresponding to the pigment concentration distribution determination range with the highest proportion. The method for estimating a light source based on the distribution of skin pigment components as described in feature 1.

18. The light source estimation unit determines the type of the estimated target light source based on the proportion of pixels located within each pigment density distribution determination range for the target image, The light source estimation unit determines the weight corresponding to each pigment density distribution determination range based on the proportion of pixels located within each pigment density distribution determination range for the target image, The light source estimation unit determines the light source color of the estimated target light source by weighting the light source colors of the measurement light sources corresponding to the multiple pigment concentration distribution determination ranges based on weights corresponding to the multiple pigment concentration distribution determination ranges. including, The method for estimating a light source based on the distribution of skin pigment components as described in feature 1.

19. An image processing method, The type of light source to be estimated is determined by the method for estimating a light source based on the distribution of skin pigment components described in any one of claims 1 to 18, Based on the determined type of the estimated target light source, the target white balance gain and / or target color correction value are determined. Processing an image acquired under the estimated target light source based on the target white balance gain and / or the target color correction value, including, An image processing method characterized by the following:

20. An imaging device, It includes an image acquisition unit, a judgment range calculation unit, and a light source estimation unit. The image acquisition unit is used to acquire a target image of the subject under the estimated target light source and to determine the skin region of the target image. The determination range calculation unit is used to determine the dye concentration distribution determination range for multiple measurement light sources of different types, separately for each of the multiple measurement light sources, in order to obtain multiple dye concentration distribution determination ranges corresponding to the multiple measurement light sources, with each measurement light source corresponding to one dye concentration distribution determination range. The light source estimation unit converts the pixel values ​​of the skin region of the target image into density values ​​to obtain skin pixel distribution points in density space for the target image, projects the skin pixel distribution points of the target image onto a unit plane along the direction of the shading component vector to obtain pigment component distribution points for the target image, determines the proportion of pixels located within each pigment density distribution determination range for the target image based on the pigment component distribution points of the target image and the pigment density distribution determination range corresponding to each measurement light source, and determines the type of the estimated target light source based on the proportion of pixels located within each pigment density distribution determination range for the target image. An imaging device characterized by the following features.

21. The imaging device further includes a color correction unit, The color correction unit determines a target white balance gain and / or a target color correction value based on the type of the estimated target light source determined by the light source estimation unit, and is used to process an image acquired under the estimated target light source based on the target white balance gain and / or the target color correction value. The imaging apparatus according to feature 20.

22. A computer-readable storage medium, The computer-readable storage medium stores a plurality of computer instructions, and when the computer instructions are loaded into a processor and executed, it performs the light source estimation method based on the distribution of skin pigment components described in any one of claims 1 to 18.

23. A computer-readable storage medium, The computer-readable storage medium stores a plurality of computer instructions, and when the computer instructions are loaded into a processor and executed, it performs the image processing method described in claim 19.

Images