Skin detection method, apparatus, face mask, and storage medium

Abstract

The application relates to a skin detection method, device, mask and storage medium. The method comprises the following steps: in the case that a face is detected wearing a mask, a first color matrix obtained by shooting a complete facial region of the face through a camera in the mask is acquired; a second color matrix measured by a color sensor in the mask on a local facial region in the complete facial region is acquired; based on the first color matrix and the second color matrix, a local difference of the camera and the color sensor at the local facial region is determined; the first color matrix is color corrected based on the local difference, and a corrected color matrix is obtained; and the skin color of the face is determined according to the corrected color matrix. The method can improve the accuracy of the detected skin color of the face.

Description

Technical Field

[0001] This application relates to the field of skin detection technology, and in particular to a skin detection method, device, mask, and storage medium. Background Technology

[0002] With the development of computer technology, skin detection technology has emerged. This technology can detect human skin color, and the results can be applied to various fields, such as diagnosing skin problems, skin care, and skin diseases. Current technology typically involves capturing skin images with a camera and then analyzing the image colors to determine the skin color.

[0003] However, current methods of determining skin color by capturing images of skin with a camera and analyzing the colors in those images suffer from low accuracy in determining skin color. Summary of the Invention

[0004] Therefore, it is necessary to provide a skin detection method, device, mask, and computer-readable storage medium that can improve accuracy in addressing the aforementioned technical problems.

[0005] In a first aspect, this application provides a skin detection method, comprising:

[0006] When a face mask is detected, a first color matrix is ​​obtained after the complete facial area of ​​the face is captured by the camera in the mask.

[0007] Obtain a second color matrix measured by a color sensor in the mask for a local facial region within the full facial region;

[0008] Based on the first color matrix and the second color matrix, determine the local differences between the camera and the color sensor in the local facial region;

[0009] Based on the local differences, the first color matrix is ​​color-corrected to obtain a corrected color matrix;

[0010] The skin color of the face is determined based on the corrected color matrix.

[0011] Secondly, this application also provides a skin detection device, comprising:

[0012] The acquisition module is used to acquire, when a face mask is detected, a first color matrix obtained by capturing the complete facial area of ​​the face through a camera in the mask; and to acquire a second color matrix measured by a color sensor in the mask of a local facial area within the complete facial area.

[0013] A local difference determination module is used to determine the local differences between the camera and the color sensor in the local facial region based on the first color matrix and the second color matrix;

[0014] The correction module is used to perform color correction on the first color matrix based on the local differences to obtain a corrected color matrix;

[0015] A skin color determination module is used to determine the skin color of the human face based on the corrected color matrix.

[0016] Thirdly, this application also provides a face mask, including a memory and a processor, and further including a camera for capturing the complete facial region of a face and a color sensor for capturing a partial facial region of a face. The memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0017] When a face mask is detected, a first color matrix is ​​obtained after the complete facial area of ​​the face is captured by the camera in the mask.

[0018] Obtain a second color matrix measured by a color sensor in the mask for a local facial region within the full facial region;

[0019] Based on the first color matrix and the second color matrix, determine the local differences between the camera and the color sensor in the local facial region;

[0020] Based on the local differences, the first color matrix is ​​color-corrected to obtain a corrected color matrix;

[0021] The skin color of the face is determined based on the corrected color matrix.

[0022] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0023] When a face mask is detected, a first color matrix is ​​obtained after the complete facial area of ​​the face is captured by the camera in the mask.

[0024] Obtain a second color matrix measured by a color sensor in the mask for a local facial region within the full facial region;

[0025] Based on the first color matrix and the second color matrix, determine the local differences between the camera and the color sensor in the local facial region;

[0026] Based on the local differences, the first color matrix is ​​color-corrected to obtain a corrected color matrix;

[0027] The skin color of the face is determined based on the corrected color matrix.

[0028] The aforementioned skin detection method, device, mask, and storage medium include a camera and a color sensor in the mask. Since the camera's image quality is significantly affected by ambient light, the skin color determined by the first color matrix obtained after capturing a complete facial area may deviate considerably from the actual skin color. The color sensor, specifically designed for color measurement, has higher accuracy than the camera. Therefore, in localized facial areas, the second color matrix is ​​more accurate than the first color matrix. Because the color sensor only measures localized facial areas, fewer sensors are needed, saving hardware resources. Furthermore, by using the more accurate second color matrix and the less accurate first color matrix to determine local differences, and then performing color correction on the first color matrix based on these local differences, the accuracy of the first color matrix can be improved, thereby enhancing the accuracy of the detected facial skin color. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating a skin detection method in one embodiment;

[0031] Figure 2 This is a flowchart illustrating the steps of determining the local differences between the camera and the color sensor in a local facial region based on the first color matrix and the second color matrix in one embodiment.

[0032] Figure 3 This is a structural block diagram of the skin detection device in one embodiment;

[0033] Figure 4 This is a diagram of the internal structure of the face mask in one embodiment. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] In one embodiment, such as Figure 1 As shown, a skin detection method is provided. This embodiment illustrates the application of this method to a face mask. It is understood that this method can also be applied to a system including a face mask and a computer device, and implemented through the interaction between the face mask and the computer device. The computer device can be a smartphone, tablet, laptop, desktop computer, or server. In this embodiment, the method includes the following steps:

[0036] Step 102: When a face mask is detected, obtain the first color matrix obtained after the camera in the mask captures the complete facial area of ​​the face.

[0037] The mask is a device used for skin detection. One or more cameras may be installed in the mask. The complete facial area is the entire area of ​​the face, specifically the area within the facial contour lines that identify the face when viewed from a frontal view. The mask may be electrically connected to the cameras and a color sensor. The mask may be equipped with a display unit to display the skin color of the face. In another embodiment, the mask may establish a communication connection with a computer device having a display unit, transmitting the skin color of the face to and displaying it on the computer device. The camera is a device capable of capturing images, used to acquire the complete facial area.

[0038] The first color matrix is ​​a matrix representation of a face image captured by a camera covering the entire facial area. Specifically, the first color matrix can be a color matrix in RGB color mode, where the color value of each pixel can be the channel pixel value of its R (Red), G (Green), and B (Blue) channels. In RGB color mode, each pixel can include the channel pixel values ​​of each channel. A color matrix in RGB color mode can include an R channel matrix, a G channel matrix, and a B channel matrix.

[0039] In one embodiment, the mask may contain only one camera. In this embodiment, when a face is detected wearing the mask, the mask can control the camera within it to capture an image of the entire facial area, obtaining a first color matrix after the camera captures the face. The camera can be positioned in the center of the mask, and when the face is wearing the mask, it can capture an image of the face from a frontal viewpoint, obtaining a frontal view image of the face. The viewpoint is the angle from which the face is observed. The viewpoint can be a frontal viewpoint (observing the face from the front) or a side viewpoint, which can be a left-side view or a right-side view.

[0040] In one embodiment, the mask may be equipped with multiple cameras, each used to capture images of a face from different perspectives. In this embodiment, when a face is detected wearing the mask, the mask can control the multiple cameras within it to capture images of the face separately, obtaining facial images captured by each camera from different perspectives. The facial images captured by the multiple cameras are then fused to obtain a fused facial image from a frontal viewpoint. The fused facial image is represented as a matrix to obtain a first color matrix.

[0041] Step 104: Obtain the second color matrix measured by the color sensor in the mask for a local facial region within the full facial region.

[0042] The color sensor identifies the color of an object's surface. Based on the theory of primary colors and the photoelectric effect, the color sensor determines the color of an object's surface by measuring the intensity and frequency of light. A local facial region is the area measured by the color sensor within the complete facial region. A local facial region is a specific area within the complete facial region; for example, it could be located in the cheek area of ​​a face. The size of the local facial region matches the size of the area that the color sensor can measure.

[0043] The mask may contain only one color sensor, in which case there may be only one local facial region and one second color matrix. Alternatively, the mask may contain multiple color sensors, in which case there may be multiple local facial regions, with each color sensor measuring a different local facial region, and each color sensor measuring a second color matrix.

[0044] The second color matrix is ​​a color matrix in RGB color mode. The color value of each pixel in the color matrix can be the channel pixel value of the R channel, G channel and B channel of that pixel.

[0045] In one embodiment, while the mask controls the camera to capture the entire facial area of ​​a person, it also controls the color sensor in the mask to measure a local facial area within the entire facial area, thereby obtaining a second color matrix measured by the color sensor.

[0046] Step 106: Based on the first color matrix and the second color matrix, determine the local differences between the camera and the color sensor in the local facial region.

[0047] Local differences characterize the differences in color recognition accuracy between the camera and the color sensor in local facial regions. Specifically, local differences can be represented in matrix form.

[0048] In one embodiment, the mask can determine a sub-color matrix in a first color matrix that matches a local facial region, and based on the sub-color matrix and a second color matrix, determine the local differences between the camera and the color sensor at the local facial region. The sub-color matrix is ​​the color matrix of the first color matrix at the local facial region; that is, the sub-color matrix is ​​the color matrix in the first color matrix that is the same as the region measured by the second color matrix.

[0049] In one embodiment, the mask can perform matrix subtraction on the second color matrix and the sub-color matrix to obtain a local difference matrix, which is then used to determine the local differences between the camera and the color sensor in a local facial region.

[0050] Step 108: Perform color correction on the first color matrix based on local differences to obtain the corrected color matrix.

[0051] The corrected color matrix is ​​the color matrix obtained by correcting the first color matrix. Color correction is a process that makes the colors in the first color matrix closer to reality.

[0052] In one embodiment, the mask may contain only one color sensor. In this embodiment, the mask can expand local differences into complete differences that match the entire facial area, and perform color correction on the first color matrix based on the complete differences to obtain a corrected color matrix.

[0053] In one embodiment, the mask may include multiple color sensors, and the complete facial area is composed of multiple partitioned facial areas. Each partitioned facial area contains local facial areas, and the size of the partitioned facial area is larger than the size of the local facial areas. The second color matrix measured by each of the multiple color sensors may be obtained by measuring the local facial areas in different partitioned facial areas within the complete facial area. The local difference corresponding to each local facial area is determined based on the first color matrix and the second color matrix corresponding to that local facial area. In this embodiment, the mask may, for each local facial area, extend the local difference corresponding to the targeted local facial area to the difference of the partitioned facial area corresponding to the partitioned facial area where the targeted local facial area is located. Based on the differences of the partitioned facial areas corresponding to each of the multiple partitioned facial areas, the first color matrix is ​​color-corrected to obtain a corrected color matrix. The partitioned facial areas are areas corresponding to different facial parts within the complete facial area; for example, the multiple partitioned facial areas may be the cheek area, mouth area, forehead area, nose area, and eye area.

[0054] Step 110: Determine the skin color of the face based on the corrected color matrix.

[0055] Skin color refers to the color reflected by pigments such as melanin, heme, and carotene in the epidermis of human skin. Skin color can be one of the basic skin colors, such as black, brown, yellow, and white. Skin color can also be further graded from the basic skin color. For example, after grading yellow according to its depth, skin color can be categorized as light yellow, medium yellow, etc.

[0056] In one embodiment, the corrected color matrix can be a color matrix in RGB color mode. In this embodiment, the mask can convert the corrected color matrix from RGB color mode to LAB color mode to obtain a color matrix in LAB color mode, and determine the skin color of the face based on the color matrix in LAB color mode.

[0057] In the aforementioned skin detection method, the mask is equipped with a camera and a color sensor. Since the camera's image quality is significantly affected by ambient light, the skin color determined by the first color matrix obtained after capturing the entire facial area may deviate considerably from the actual skin color. The color sensor, however, is specifically designed for color measurement, and its accuracy is higher than that of the camera. Therefore, in localized facial areas, the second color matrix is ​​more accurate than the first color matrix. Because the color sensor only measures localized facial areas, fewer sensors are needed, saving hardware resources. Furthermore, by using the more accurate second color matrix and the less accurate first color matrix to determine local differences, and then performing color correction on the first color matrix based on these local differences, the accuracy of the first color matrix can be improved, thereby enhancing the accuracy of the detected facial skin color.

[0058] In one embodiment, such as Figure 2 As shown, step 106 includes steps 202 to 204. Wherein:

[0059] Step 202: Determine the sub-color matrix in the first color matrix that matches the local facial region.

[0060] Step 204: Based on the second color matrix, the sub-color matrix, and the color conversion matrix pre-configured for the camera and color sensor, determine the local differences between the camera and color sensor in the local facial region.

[0061] The color conversion matrix is ​​used to convert the color matrix measured by the color sensor into a matrix that can be used to perform operations with the color matrix captured by the camera.

[0062] In this embodiment, by determining the sub-color matrix in the first color matrix that matches the local facial region, i.e., determining the color matrix of the camera at the local facial region, and then based on the second color matrix, the sub-color matrix, and the color conversion matrix, it is convenient to determine the local differences between the camera and the color sensor at the local facial region. Combined with subsequent steps, the accuracy of detecting facial skin color can be improved.

[0063] In one embodiment, the mask can determine a matrix region in the first color matrix that matches a local facial region, and the matrix in that matrix region is defined as a sub-color matrix in the first color matrix that matches the local facial region. The relative position between the camera and the color sensor can be preset. When the first color matrix captured by the camera is acquired, the matrix region in the first color matrix that matches the local facial region measured by the color sensor can be determined based on the relative position between the camera and the color sensor.

[0064] In one embodiment, the mask can perform matrix multiplication on a second color matrix and a color conversion matrix pre-configured for the camera and color sensor to obtain a converted color matrix. Then, it can perform matrix subtraction on the converted color matrix and a sub-color matrix to obtain a local difference matrix. This local difference matrix represents the local differences between the camera and the color sensor in local facial regions. The second color matrix, sub-color matrix, color conversion matrix, converted color matrix, and local difference matrix can all be color matrices in RGB color mode. In RGB color mode, the matrix multiplication and subtraction operations can be performed using channel matrices of different color matrices in the same channel.

[0065] In one embodiment, step 202 includes: when there are dark pixel regions in the first color matrix and the proportion of pixels in the dark pixel regions in the sub-color matrix exceeds a preset proportion, performing color equalization processing on the color values ​​of the pixels in the sub-color matrix and the second color matrix respectively; the overall pixel color of the dark pixel region is darker than the overall pixel color in the neighborhood of the dark pixel region in the first color matrix; based on the color conversion matrix pre-configured for the camera and color sensor, the color-equalized second color matrix, and the color-equalized sub-color matrix, determining the local differences between the camera and color sensor in the local facial region.

[0066] The preset percentage is a pre-set ratio value. The preset percentage can be 5%, 10%, 50%, or other values. Color balance processing balances the overall color of the color matrix, ensuring a uniform distribution of color depth.

[0067] Dark pixel regions are areas in the first color matrix that are darker than the overall pixel color in their neighborhood. The size of the facial region corresponding to a dark pixel region in a human face can be larger than a preset size, such as 2 square millimeters or 3 square millimeters. When the size of the facial region is smaller than the preset size, it can be considered a region where color influence is negligible; for example, this facial region could be the area containing a small mole on the face.

[0068] The neighborhood of a dark pixel region is the area surrounding a dark pixel region in the first color matrix. Specifically, the neighborhood of a dark pixel region can consist of pixel positions outside the dark pixel region and within the neighborhood of pixels located at the edges of the dark pixel region. The neighborhood of a pixel position can be a 4-neighborhood, an 8-neighborhood, or others. The presence of dark pixel regions in the first color matrix indicates uneven skin color across the entire face. If the proportion of pixels in dark pixel regions in the sub-color matrix exceeds a preset proportion, it indicates that there is uneven color in a local facial area, and the dark region accounts for a certain percentage.

[0069] When comparing the overall pixel color intensity of different regions in a color matrix, the overall color brightness between different regions can be compared. Specifically, if the overall color brightness of one region is darker than that of another region, it indicates that the overall pixel color of the one region is darker than that of the other region. A region in the color matrix can be an entire pixel region or a neighborhood of an entire pixel region.

[0070] The overall color brightness of a region in the color matrix can be the representative brightness value of that region. When the representative brightness value of one region is less than the representative brightness value of another region, and the absolute value of the difference between the representative brightness values ​​of one region and the other region is greater than a preset brightness difference value, it can be determined that the overall color brightness of one region is darker than that of the other region.

[0071] The representative brightness value can be the average brightness value of all pixels in the region, the mode of the brightness values ​​of all pixels in the region, or the brightness value that represents the brightest brightness value in the region. The average brightness value can be the arithmetic mean brightness value, that is, the arithmetic average of the brightness values ​​of all pixels in the region; the average brightness value can also be a weighted average brightness value, in which pixels at different positions in the region have different weights, with pixels closer to the center of the region having a greater weight.

[0072] The brightness value of a pixel can be the pixel value in the luminance channel after converting the pixel from RGB color mode to a color mode that includes a luminance channel. The color mode including the luminance channel can be HSV color mode, which includes the H (Hue) channel, S (Saturation) channel, and V (Value) channel; or it can be LAB color mode, which includes the L (Lightness) channel, A (Green-Red Axis) channel, and B (Blue-Yellow Axis) channel. In LAB color mode, the preset brightness difference value can be 20, 30, 40, or others.

[0073] In this embodiment, when the first color matrix contains dark pixel regions and the proportion of pixels in the dark pixel regions in the sub-color matrix exceeds a preset proportion, it indicates that there is uneven color in a local facial area and the dark regions have reached a certain proportion. At this time, color equalization processing is performed on the color values ​​of the pixels in the sub-color matrix and the second color matrix respectively. Subsequently, the local differences are determined based on the color transformation matrix, the second color matrix after color equalization processing, and the sub-color matrix after color equalization processing. This can avoid the error caused by the dark regions in the local facial area and improve the accuracy of detecting facial skin color.

[0074] In one embodiment, both the sub-color matrix and the second color matrix are color matrices in RGB color mode. Color equalization processing of the pixel color values ​​of the sub-color matrix and the second color matrix respectively includes: determining the first dark pixel quantity ratio and the first light pixel quantity ratio of pixels in the dark pixel region of the sub-color matrix; for each channel of the sub-color matrix in RGB color mode, determining the first dark channel average value and the first light channel average value of the sub-color matrix in the targeted channel; summing the product of the first dark channel average value and the first dark pixel quantity ratio, and the product of the first light channel average value and the first light pixel quantity ratio, to obtain the first channel pixel representative value of the sub-color matrix in the targeted channel; and determining the first channel pixel representative value as the channel pixel value of each pixel in the sub-color matrix in the targeted channel, thereby performing color equalization processing on the pixel color values ​​of the sub-color matrix.

[0075] The first dark channel average value of the targeted channel is the average pixel channel value of pixels in the dark pixel region of the sub-color matrix in the targeted channel. The first light channel average value of the targeted channel is the average pixel channel value of pixels in the non-dark pixel region of the sub-color matrix in the targeted channel.

[0076] In one embodiment, both the sub-color matrix and the second color matrix are color matrices in RGB color mode. Color equalization processing of the pixel color values ​​of the sub-color matrix and the second color matrix respectively includes: determining local dark pixel regions in the second color matrix that match the dark pixel regions of the first color matrix; determining the proportion of second dark pixels in the local dark pixel regions and the proportion of second light pixels not in the local dark pixel regions in the second color matrix; for each channel of the second color matrix in RGB color mode, determining the average value of the second dark channel and the average value of the second light channel in the targeted channel; summing the product of the average value of the second dark channel and the proportion of second dark pixels, and the product of the average value of the second light channel and the proportion of second light pixels, to obtain the representative value of the second channel pixel in the targeted channel; and determining the representative value of the second channel pixel as the channel pixel value of each pixel in the second color matrix in the targeted channel, thereby performing color equalization processing on the pixel color values ​​of the second color matrix.

[0077] In this context, the locally dark pixel regions in the second color matrix occupy the same pixel positions in the sub-color matrix as the locally dark pixel regions in the first color matrix. The average value of the second dark channel for a given channel is the average pixel channel value of the pixels in the locally dark pixel regions within the second color matrix in the given channel. The average value of the second light channel for a given channel is the average pixel channel value of the pixels in the second color matrix that are not in the locally dark pixel regions in the given channel.

[0078] In one embodiment, the skin detection method further includes the following steps: determining a sub-color matrix in the first color matrix that matches a local facial region; when the first color matrix contains dark pixel regions and the proportion of pixels in the dark pixel regions in the sub-color matrix exceeds a preset proportion, while keeping the relative positions between the camera and the color sensor unchanged, adjusting the positions of the camera and the color sensor in the mask, performing the steps of obtaining the first color matrix obtained after capturing the complete facial region of the face through the camera in the mask, and obtaining the second color matrix measured by the color sensor in the mask for a local facial region in the complete facial region.

[0079] The relative positions between the camera and the color sensor can be preset. For example, during mask calibration, the relative positions between the camera and the color sensor are set and calibrated, and then kept constant after calibration. The camera and color sensor can be configured as a complete color detection module, which can be fixed to a movable structural sliding component of the mask. The mask can adjust the position of the color detection module via the structural sliding component, thereby adjusting the positions of the camera and color sensor within the mask while maintaining their relative positions.

[0080] In this embodiment, when there are dark pixel areas in the first color matrix and the proportion of pixels in the dark pixel areas in the sub-color matrix exceeds a preset proportion, it indicates that there is uneven color in a local facial area and the dark area reaches a certain proportion. At this time, the position of the camera and color sensor in the mask is adjusted so that the local facial area can be updated. The local differences can be determined in the future, avoiding the error caused by the dark area in the local facial area, and improving the accuracy of the detected facial skin color.

[0081] In one embodiment, when there are dark pixel areas in the first color matrix and the proportion of pixels in the dark pixel areas in the sub-color matrix exceeds a preset proportion, the mask can adjust the position of the camera and the color sensor in the mask while keeping the relative position between the camera and the color sensor unchanged, and execute steps 102 and 104, and steps 106 to 110.

[0082] In one embodiment, when there are dark pixel areas in the first color matrix and the proportion of pixels in the dark pixel areas in the sub-color matrix exceeds a preset proportion, the mask can adjust the position of the camera and the color sensor in the mask while keeping the relative position between the camera and the color sensor unchanged. After executing steps 102 and 104, the sub-color matrix that matches the local facial area in the first color matrix is ​​determined. When the proportion of pixels in the dark pixel areas in the sub-color matrix does not exceed the preset proportion, steps 106 to 110 are continued.

[0083] In one embodiment, step 108 includes: obtaining a pre-configured difference expansion matrix for the camera; expanding the local differences to a complete difference that matches the complete facial region based on the difference expansion matrix; and performing color correction on the first color matrix according to the complete differences to obtain a corrected color matrix.

[0084] The difference expansion matrix is ​​used to extend local differences to a complete difference that matches the entire facial region. The complete difference characterizes the difference in color recognition accuracy between the camera and the color sensor over the entire facial region. Specifically, the complete difference can be represented in matrix form.

[0085] In this embodiment, by using a difference extension matrix, local differences can be extended to the differences of the entire face, thereby facilitating color correction of the first color matrix based on the complete differences and obtaining a corrected color matrix with higher accuracy than the first color matrix, which can improve the accuracy of detecting facial skin color.

[0086] In one embodiment, the local difference can be a local difference matrix. In this embodiment, the mask can perform a Kronecker product operation on the local difference matrix and the difference extension matrix to obtain a complete difference matrix that matches the complete facial region, and the complete difference matrix is ​​determined as the complete difference that matches the complete facial region.

[0087] In one embodiment, the mask can obtain an intermediate operation matrix by performing matrix subtraction between the complete difference matrix and the first color matrix. The absolute value of the color value of each pixel in the intermediate operation matrix is ​​then taken to obtain a corrected color matrix. Both the complete difference matrix and the first color matrix can be color matrices in RGB color mode. The intermediate operation matrix includes a channel matrix obtained by performing matrix subtraction between the complete difference matrix and the first color matrix in each channel. Taking the absolute value of the color value of each pixel in the intermediate operation matrix involves taking the absolute value of the channel pixel value of each pixel in the intermediate operation matrix in each channel.

[0088] In one embodiment, the color correction matrix is ​​a color matrix in RGB color mode, and step 110 includes: determining the pixels belonging to the face region in the color correction matrix; converting the pixels belonging to the face region from RGB color mode to LAB color mode respectively to obtain the channel pixel values ​​of the pixels belonging to the face region in each channel in LAB color mode; determining the representative pixel values ​​of the pixels belonging to the face region in each channel based on the channel pixel values ​​of the pixels belonging to the face region in each channel in LAB color mode; determining the preset pixel value range matched by the representative pixel values ​​in each channel; and determining the skin color of the face according to the preset pixel value range matched by the representative pixel values ​​in each channel.

[0089] The LAB color mode, defined by the International Commission on Illumination (ICI), theoretically encompasses all colors visible to the human eye. In LAB, the L channel is the luminance channel, with pixel values ​​ranging from 0 to 100. Higher values ​​indicate brighter skin, which can be interpreted as whiter skin. The A channel is the red-green hue channel, with pixel values ​​ranging from 127 to -128, representing the range from red to green; higher values ​​indicate redder skin. The B channel is the blue-yellow hue channel, with pixel values ​​ranging from 127 to -128, representing the range from yellow to blue; higher values ​​indicate redder skin. The pixel values ​​of each channel in LAB can serve as a numerical representation of skin color on a face. After determining a person's skin color, a mask can be used to display that color, along with the pixel values ​​of the corresponding facial area in LAB mode. This allows users to perceive skin color not only through its numerical representation but also through the visual representation itself.

[0090] The representative pixel values ​​for each channel of the face region can be the average of the corresponding channel pixel values ​​in the LAB color mode, or the average of the remaining pixels in the face region after excluding dark pixels, in the corresponding channel of the LAB color mode. This average can be an arithmetic mean or a weighted average. Specifically, different weights can be assigned to pixels in different facial regions; for example, higher weights can be assigned to the cheeks, medium weights to the mouth and nose, and lower weights to the forehead and eye areas.

[0091] The skin color of a face can be one of several preset skin colors, each corresponding to a set of preset pixel value ranges for each channel in LAB color mode. From darkest to lightest, preset skin colors can be black, brown, light yellow (three shades of white), light yellow (two shades of white), light yellow (one shade of white), or white. Understandably, the value range for each channel in LAB color mode is quite wide; therefore, preset skin colors can be further subdivided into finer granularities. By assigning preset skin colors to different sets of preset pixel value ranges, the accuracy of detected facial skin color can be further improved.

[0092] The preset pixel value range that a pixel value matches is the preset pixel value range in which the pixel value belongs. For example, when the preset skin color is yellow-white, the preset pixel value range corresponding to each channel under a set of LAB channels could be 10 to 20 for the L channel, 0 to -20 for the A channel, and 20 to 0 for the B channel.

[0093] In this embodiment, the pixels belonging to the face region in the correction color matrix are converted from RGB color mode to LAB color mode, thereby determining the representative pixel value of the face region in each channel. By matching the preset pixel value range of the representative pixel value in each channel, the skin color of the face is determined. The skin color of the face can be quantitatively determined, which can further improve the accuracy of the detected face skin color when the skin color is divided into finer granularities.

[0094] In one embodiment, when there are no dark pixel regions in the corrected color matrix, the mask can determine the arithmetic mean of the channel pixel values ​​of the pixels belonging to the face region in each channel under the LAB color mode as the representative pixel value of the face region in each channel.

[0095] In one embodiment, when there are dark pixel regions in the color correction matrix, and the absolute value of the difference between the representative brightness value of the dark pixel region and the representative brightness value in the neighborhood of the dark pixel region is greater than a first preset brightness difference value, the mask can determine the weighted average value of the channel pixel values ​​of the pixels belonging to the face region in each channel in LAB color mode according to the weights of the different facial regions, and determine the weighted average value as the representative pixel value of the pixels of the face region in each channel.

[0096] In one embodiment, when a dark pixel region exists in the corrected color matrix, and the absolute value of the difference between the representative brightness value of the dark pixel region and the representative brightness value in the neighborhood of the dark pixel region is greater than a second preset brightness difference value, the mask determines a skin treatment strategy for the face. Under this skin treatment strategy, the first strategy is used to treat areas of the face that match the dark pixel region, and the second strategy is used to treat areas of the face that do not match the dark pixel region. The first preset brightness difference value is greater than a preset brightness difference value, and the second preset brightness difference value is greater than the first preset brightness difference value. The first strategy may be treatment using a phototherapy unit or phototherapy device in the mask at a first preset power, and the second strategy may be treatment using a phototherapy unit or phototherapy device in the mask at a second preset power, where the first preset power is greater than the second preset power.

[0097] In one embodiment, prior to the skin detection method described above, the mask is calibrated. Specifically, the mask can be placed under a preset light source. The mask controls the camera to capture images of a standard color chart (such as a gray chart or a 24-color chart) to obtain a first color chart color matrix, and controls the color sensor to measure the standard color chart to obtain a second color chart color matrix. The camera captures images of the complete facial region of the sample face to obtain a first sample color matrix, and the color sensor measures a local facial region within the complete facial region of the sample face to obtain a second sample color matrix. A subsample color matrix matching the local facial region of the sample face in the first sample color matrix is ​​determined. Based on the subsample color matrix and the second sample color matrix, a color transformation matrix is ​​determined such that matrix multiplication of the second sample color matrix and the color transformation matrix yields the subsample color matrix. This color transformation matrix is ​​then defined as a pre-configured color transformation matrix for the camera and color sensor. Based on the first sample color matrix and the subsample color matrix, a difference expansion matrix is ​​determined such that Kronecker product of the subsample color matrix and the difference expansion matrix yields the first sample color matrix. This difference expansion matrix is ​​then defined as a difference expansion matrix configured for the camera and stored.

[0098] The first sample color matrix is ​​obtained by calibrating the original color matrix captured by the camera using a first color chart. The second sample color matrix is ​​obtained by calibrating the original color matrix measured by the color sensor using a second color chart. After the mask undergoes the above calibration, the preset light source and the relative position between the camera and the color sensor must remain unchanged during skin detection.

[0099] In one embodiment, the above-described skin detection method may specifically include the following steps.

[0100] When a face mask is detected, the mask can acquire a first color matrix obtained by capturing the complete facial area of ​​the face through a camera in the mask, and a second color matrix obtained by measuring a local facial area within the complete facial area through a color sensor in the mask.

[0101] The mask can determine the sub-color matrix in the first color matrix that matches the local facial region; when the first color matrix does not have a dark pixel region, or when the first color matrix has a dark pixel region and the proportion of pixels in the dark pixel region in the sub-color matrix does not exceed a preset proportion, the mask can perform matrix multiplication operation on the second color matrix and the color conversion matrix pre-configured for the camera and color sensor to obtain the converted color matrix, and perform matrix subtraction operation on the converted color matrix and the sub-color matrix to obtain the local difference matrix.

[0102] When the first color matrix contains dark pixel regions, and the proportion of pixels in the dark pixel regions in the sub-color matrix exceeds a preset proportion, the mask can perform color equalization processing on the color values ​​of the pixels in the sub-color matrix and the second color matrix respectively; the mask can perform matrix multiplication operation on the second color matrix after color equalization and the color transformation matrix to obtain the transformed color matrix, and perform matrix subtraction operation on the transformed color matrix and the sub-color matrix after color equalization to obtain the local difference matrix.

[0103] The mask can obtain a pre-configured difference extension matrix for the camera, perform a Kronecker product operation on the local difference matrix and the difference extension matrix to obtain a complete difference matrix that matches the complete facial region, perform a matrix subtraction operation on the complete difference matrix and the first color matrix to obtain an intermediate operation matrix, and take the absolute value of the color value of each pixel in the intermediate operation matrix to obtain a corrected color matrix.

[0104] The mask identifies pixels belonging to the face region in the color correction matrix; it converts these pixels from RGB color mode to LAB color mode to obtain the channel pixel values ​​of each channel in LAB color mode; based on these channel pixel values, it determines the representative pixel values ​​of each pixel in each channel; it determines the preset pixel value range that matches the representative pixel values ​​of each channel; and it determines the skin color of the face based on the preset pixel value range that matches the representative pixel values ​​of each channel.

[0105] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0106] Based on the same inventive concept, this application also provides a skin detection device for implementing the skin detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more skin detection device embodiments provided below can be found in the limitations of the skin detection method described above, and will not be repeated here.

[0107] In one embodiment, such as Figure 3 As shown, a skin detection device 300 is provided, including: an acquisition module 310, a local difference determination module 320, a correction module 330, and a skin color determination module 340, wherein:

[0108] The acquisition module 310 is used to acquire, when a face mask is detected, a first color matrix obtained by capturing the complete facial area of ​​the face through a camera in the mask; and to acquire a second color matrix measured by a color sensor in the mask of a local facial area within the complete facial area.

[0109] The local difference determination module 320 is used to determine the local differences between the camera and the color sensor in a local facial region based on the first color matrix and the second color matrix.

[0110] The correction module 330 is used to perform color correction on the first color matrix based on local differences to obtain a corrected color matrix.

[0111] The skin color determination module 340 is used to determine the skin color of a human face based on the corrected color matrix.

[0112] In one embodiment, the local difference determination module 320 is further configured to determine a sub-color matrix in the first color matrix that matches the local facial region; and to determine the local differences between the camera and the color sensor at the local facial region based on the second color matrix, the sub-color matrix, and a color conversion matrix pre-configured for the camera and the color sensor.

[0113] In one embodiment, the local difference determination module 320 is further configured to perform color equalization processing on the color values ​​of the pixels in the sub-color matrix and the second color matrix respectively when there are dark pixel regions in the first color matrix and the proportion of pixels in the dark pixel regions in the sub-color matrix exceeds a preset proportion; the overall pixel color of the dark pixel region is darker than the overall pixel color in the neighborhood of the dark pixel region in the first color matrix; and determine the local differences between the camera and the color sensor in the local facial region based on the color conversion matrix pre-configured for the camera and the color sensor, the second color matrix after color equalization processing, and the sub-color matrix after color equalization processing.

[0114] In one embodiment, the skin detection device 300 further includes an adjustment module, which is used to determine a sub-color matrix in the first color matrix that matches a local facial region; when there is a dark pixel region in the first color matrix and the proportion of pixels in the dark pixel region in the sub-color matrix exceeds a preset proportion, while keeping the relative position between the camera and the color sensor unchanged, the position of the camera and the color sensor in the mask is adjusted, and the step of obtaining the first color matrix obtained after the camera in the mask captures the complete facial region of the face, and the step of obtaining the second color matrix measured by the color sensor in the mask for a local facial region in the complete facial region.

[0115] In one embodiment, the correction module 330 is further configured to obtain a pre-configured difference extension matrix for the camera; based on the difference extension matrix, extend the local differences to the complete differences that match the complete facial region; and perform color correction on the first color matrix according to the complete differences to obtain a corrected color matrix.

[0116] In one embodiment, the corrected color matrix is ​​a color matrix in RGB color mode, and the skin color determination module 340 is further used to determine the pixels belonging to the face region in the corrected color matrix; convert the pixels belonging to the face region from RGB color mode to LAB color mode respectively to obtain the channel pixel values ​​of the pixels belonging to the face region in each channel in LAB color mode; based on the channel pixel values ​​of the pixels belonging to the face region in each channel in LAB color mode, determine the representative pixel values ​​of the pixels in the face region in each channel respectively; determine the preset pixel value range matched by the representative pixel values ​​in each channel; and determine the skin color of the face according to the preset pixel value range matched by the representative pixel values ​​in each channel.

[0117] Each module in the aforementioned skin detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor within the mask in hardware form, or stored in the mask's memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0118] In one embodiment, a face mask is provided, the internal structure of which can be shown in the figure below. Figure 4As shown, the mask includes a processor, memory, input / output interface, communication interface, display unit, and input device. It also includes a camera for capturing the complete facial area and a color sensor for capturing partial facial areas. The processor, memory, and input / output interface are connected via a system bus, as are the communication interface, display unit, input device, camera, and color sensor. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface allows the processor to exchange information with external devices. The communication interface allows wired or wireless communication with external terminals, such as via Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a skin detection method. The display unit of the mask forms a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the mask can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the mask shell, or external keyboards, touchpads, or mice, etc.

[0119] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the face mask to which the present application is applied. A specific face mask may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0120] In one embodiment, a face mask is also provided, including a memory and a processor, and further including a camera for capturing a complete facial region of a face and a color sensor for capturing a partial facial region of a face. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0121] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0123] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0124] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A skin detection method, characterized in that, The method includes: When a face mask is detected, a first color matrix is ​​obtained after the complete facial area of ​​the face is captured by the camera in the mask. Obtain a second color matrix measured by a color sensor in the mask for a local facial region within the full facial region; Based on the first color matrix and the second color matrix, determine the local differences between the camera and the color sensor in the local facial region; Based on the local differences, the first color matrix is ​​color-corrected to obtain a corrected color matrix; The skin color of the face is determined based on the corrected color matrix.

2. The method according to claim 1, characterized in that, The step of determining the local differences between the camera and the color sensor in the local facial region based on the first color matrix and the second color matrix includes: Determine the sub-color matrix in the first color matrix that matches the local facial region; Based on the second color matrix, the sub-color matrix, and the color conversion matrix pre-configured for the camera and the color sensor, the local differences between the camera and the color sensor in the local facial region are determined.

3. The method according to claim 2, characterized in that, The step of determining the local differences between the camera and the color sensor in the local facial region based on the second color matrix, the sub-color matrix, and a color conversion matrix pre-configured for the camera and the color sensor includes: When the first color matrix contains dark pixel regions, and the proportion of pixels in the dark pixel regions in the sub-color matrix exceeds a preset proportion, color equalization processing is performed on the color values ​​of the pixels in the sub-color matrix and the second color matrix respectively; the overall pixel color of the dark pixel region is darker than the overall pixel color of the neighborhood of the dark pixel region in the first color matrix. Based on the color conversion matrix, the second color matrix after color equalization, and the sub-color matrix after color equalization pre-configured for the camera and the color sensor, the local differences between the camera and the color sensor in the local facial region are determined.

4. The method according to claim 1, characterized in that, The method further includes: Determine the sub-color matrix in the first color matrix that matches the local facial region; When the first color matrix contains dark pixel regions, and the proportion of pixels in the dark pixel regions in the sub-color matrix exceeds a preset proportion, while keeping the relative positions between the camera and the color sensor unchanged, the positions of the camera and the color sensor in the mask are adjusted, and the steps of obtaining the first color matrix obtained after capturing the complete facial area of ​​the face through the camera in the mask and obtaining the second color matrix measured by the color sensor in the mask for a local facial area in the complete facial area are executed.

5. The method according to claim 1, characterized in that, The step of performing color correction on the first color matrix based on the local differences to obtain a corrected color matrix includes: Obtain the pre-configured difference extension matrix for the camera; Based on the difference expansion matrix, the local differences are expanded to match the complete facial region; The first color matrix is ​​color-corrected based on the complete difference to obtain a corrected color matrix.

6. The method according to any one of claims 1-5, characterized in that, The corrected color matrix is ​​a color matrix in RGB color mode. Determining the skin color of the face based on the corrected color matrix includes: Identify the pixels in the corrected color matrix that belong to the face region; The pixels belonging to the face region are converted from RGB color mode to LAB color mode to obtain the channel pixel values ​​of each channel of the pixels belonging to the face region in LAB color mode. Based on the channel pixel values ​​of the pixels belonging to the face region in each channel under LAB color mode, the representative pixel values ​​of the pixels in the face region in each channel are determined. Determine the preset range of pixel values ​​that the representative pixel value under each channel matches; The skin color of the face is determined based on the preset pixel value range matched by the representative pixel value under each channel.

7. A skin detection device, characterized in that, The device includes: The acquisition module is used to acquire, when a face mask is detected, a first color matrix obtained by capturing the complete facial area of ​​the face through a camera in the mask; and to acquire a second color matrix measured by a color sensor in the mask of a local facial area within the complete facial area. A local difference determination module is used to determine the local differences between the camera and the color sensor in the local facial region based on the first color matrix and the second color matrix; The correction module is used to perform color correction on the first color matrix based on the local differences to obtain a corrected color matrix; A skin color determination module is used to determine the skin color of the human face based on the corrected color matrix.

8. The apparatus according to claim 7, characterized in that, The local difference determination module is further configured to determine a sub-color matrix in the first color matrix that matches the local facial region; and to determine the local differences between the camera and the color sensor at the local facial region based on the second color matrix, the sub-color matrix, and a color conversion matrix pre-configured for the camera and the color sensor.

9. A face mask, comprising a memory and a processor, further comprising a camera for capturing a complete facial region of a face and a color sensor for capturing a partial facial region of a face, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images