A method, system, device, and medium for facial acne identification.
By combining multispectral imaging technology and deep learning models with parallel polarized light and near-infrared spectral images, an image mask is constructed for acne recognition, which solves the problem of low accuracy in existing technologies and achieves higher recognition accuracy and data support.
Patent Information
- Application Number
- CN202311522588.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-15
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-11-15
AI Technical Summary
Existing methods for identifying facial acne have low accuracy and cannot be used as a basis for medical diagnosis. Furthermore, diagnoses vary among different doctors and lack supporting data.
Multispectral imaging technology, including parallel polarized light images and near-infrared spectral images, is employed. Facial regions and features are identified through a deep learning model, an image mask is constructed, near-infrared grayscale binarization and acne recognition are performed, and a second deep learning model is used to correct misidentifications.
It improves the accuracy of acne identification, reduces false positives, provides clear data support, and enhances the reliability of diagnosis.
Smart Images

Figure CN117523634B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of facial acne recognition technology, and in particular to a method, system, device and medium for facial acne recognition. Background Technology
[0002] As people's living standards improve, more and more people are paying attention to skin care and treatment, especially facial skin problems. Among them, facial acne is a very common skin problem. Studies have shown that more than 95% of people will experience acne to varying degrees. Moreover, after the inflammation subsides, acne may leave behind acne scars and marks, which can continue to affect people's lives.
[0003] Current facial acne recognition methods primarily rely on RGB images captured by ordinary cameras, using computer image algorithms for acne identification. However, the accuracy is low, making them unsuitable as a basis for medical diagnosis, as they lack in-depth analysis of the RGB images. Furthermore, doctors often provide differing diagnoses for patients' skin problems, and they don't provide detailed annotations of the affected areas. When comparing before-and-after treatment results, it also depends on the doctor's interpretation, lacking clear data support.
[0004] In summary, existing technologies do not analyze and utilize the original images themselves, but instead directly identify acne on the original images, resulting in low recognition accuracy. Therefore, it is necessary to develop a method for facial acne recognition. Summary of the Invention
[0005] Therefore, the technical problem to be solved by the present invention is to overcome the low accuracy of acne identification methods in the prior art.
[0006] To address the aforementioned technical problems, this invention provides a method for facial acne identification, comprising:
[0007] Step S1: Acquire a multispectral image of the face, the multispectral image including a parallel polarized light image and a near-infrared spectral image;
[0008] Step S2: Identify the facial region and facial features in the parallel polarized light image;
[0009] Step S3: Construct an image mask based on the facial region and facial features, wherein the image mask includes areas where acne may grow and areas where acne does not grow;
[0010] Step S4: Using the potential acne-growing areas in the image mask as a reference, binarize the near-infrared spectral image to obtain a near-infrared grayscale binary image;
[0011] Step S5: Perform acne identification on the near-infrared grayscale binary image.
[0012] In one embodiment of the present invention, the method for identifying the facial region and facial features of the parallel polarized light image in step S2 includes:
[0013] The facial regions and facial features of the parallel polarized light image are identified using a first deep learning model, wherein the facial features include eyebrows, eyes, nose, mouth, ears, and hair.
[0014] In one embodiment of the present invention, step S3, which involves constructing an image mask based on the facial region and facial features, includes the following method:
[0015] The facial regions are labeled based on the facial features to obtain an image mask. The non-acne growth regions include the eyebrows, eyes, mouth, ears, hair, and nose wings. The acne-prone growth regions are the other areas of the face excluding the non-acne growth regions.
[0016] In one embodiment of the present invention, step S4 involves binarizing the near-infrared spectral image based on the potential acne-growing region in the image mask to obtain a near-infrared grayscale binary image. The method includes:
[0017] Convert the near-infrared spectral image into a near-infrared grayscale image;
[0018] Based on the potential acne-growing areas in the image mask, several pixels are uniformly sampled at preset pixel intervals on the near-infrared grayscale image, and the brightness of each pixel is recorded.
[0019] The sampled pixels are sorted by brightness, and the median brightness is taken as the brightness threshold.
[0020] The brightness of each pixel in the near-infrared grayscale image is compared with a brightness threshold. Pixels below the brightness threshold are recorded as 1, and pixels above the brightness threshold are recorded as 0, thus generating a near-infrared grayscale binary image.
[0021] In one embodiment of the present invention, step S5 involves acne identification of the near-infrared grayscale binary image, the method comprising:
[0022] In the near-infrared grayscale binary image, pixels marked as 1 are considered as pixels that may have acne.
[0023] The neighborhood algorithm is used to connect pixels that may cause acne to form several connected regions.
[0024] After generating connected regions, a high threshold T is set based on the number of pixels x in the connected regions.h With low threshold T l When the number of pixels x in the connected region is higher than the low threshold T l And below the high threshold T h If so, the connected region is considered to be acne;
[0025] The connected regions that may be acne are compared with the face mask. If more than a preset percentage of pixels in the connected regions that may be acne are in the face mask, the connected regions are retained and considered as acne; otherwise, the connected regions are removed.
[0026] In one embodiment of the present invention, the preset percentage is 60%.
[0027] In one embodiment of the present invention, after step S5, a second acne identification is performed on the acne identification result of step S5 on the parallel polarized light image, the method comprising:
[0028] Based on the connected regions identified as acne in step S5, the bounding rectangles of the connected regions identified as acne are cropped at the same position in the parallel polarized light image and used as training sample data. The training sample data is then used to train the second deep learning model.
[0029] The trained second deep learning model is used to perform a second acne identification on the parallel polarized light image to remove the erroneous acne identified in step S5.
[0030] To address the aforementioned technical problems, the present invention provides a facial acne recognition system, comprising:
[0031] Acquisition module: used to acquire multispectral images of a face, including parallel polarized light images and near-infrared spectral images;
[0032] First recognition module: used to recognize the facial region and facial features of the parallel polarized light image;
[0033] Construction module: used to construct an image mask based on the facial region and facial features, wherein the image mask includes regions where acne may grow and regions where acne does not grow;
[0034] Binarization module: Used to binarize the near-infrared spectral image based on the possible acne-growing areas in the image mask to obtain a near-infrared grayscale binary image;
[0035] The second recognition module is used to identify acne from the near-infrared grayscale binary image.
[0036] To address the aforementioned technical problems, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the aforementioned facial acne recognition method.
[0037] To address the aforementioned technical problems, the present invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned facial acne recognition method.
[0038] The technical solution of the present invention has the following advantages compared with the prior art:
[0039] This invention is the first to propose the use of multispectral images for acne identification, including using parallel polarized light to identify facial regions and feature points, binarizing near-infrared spectral images, then finding connected regions to identify the locations where acne may occur, thus completing the first acne identification.
[0040] This invention utilizes parallel polarized light images combined with a deep learning model to perform a second identification of potential acne locations, thereby reducing false identifications in the first identification. Attached Figure Description
[0041] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.
[0042] Figure 1 This is a flowchart of the method of the present invention;
[0043] Figure 2 This is a schematic diagram of the mask image in an embodiment of the present invention;
[0044] Figure 3 This is a comparison image of parallel polarized light and near-infrared spectral imaging of acne in an embodiment of the present invention;
[0045] Figure 4 This is a comparison chart of the results of the first and second acne identifications in an embodiment of the present invention. Detailed Implementation
[0046] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.
[0047] Example 1
[0048] Reference Figure 1 As shown, this invention relates to a method for identifying facial acne, comprising:
[0049] Step S1: Acquire a multispectral image of the face, the multispectral image including a parallel polarized light image and a near-infrared spectral image;
[0050] Step S2: Identify the facial region and facial features in the parallel polarized light image;
[0051] Step S3: Construct an image mask based on the facial region and facial features, wherein the image mask includes areas where acne may grow and areas where acne does not grow;
[0052] Step S4: Using the potential acne-growing areas in the image mask as a reference, binarize the near-infrared spectral image to obtain a near-infrared grayscale binary image;
[0053] Step S5: Perform acne identification on the near-infrared grayscale binary image;
[0054] Step S6: Based on the first acne identification, the parallel polarized light image is re-identified to remove the misidentification from the first identification.
[0055] This invention is the first to propose using multispectral images for acne identification. This includes using parallel polarized light to identify facial regions and feature points, binarizing the near-infrared spectral image, then searching for connected regions to identify potential acne locations, thus completing the first acne identification. This invention further utilizes parallel polarized light images combined with a deep learning model to perform a second identification of potential acne locations, reducing false positives in the first identification.
[0056] The following is a detailed description of this embodiment:
[0057] When diagnosing facial skin problems, doctors often use a multispectral skin image analyzer to photograph the face. The multispectral skin image analyzer acquires multispectral images, including parallel polarized light images / channels, cross-polarized light images / channels, near-infrared spectral images / channels, brown spectral images / channels, and ultraviolet spectral images / channels. This embodiment considers the characteristics of facial acne and uses parallel polarized light images and near-infrared spectral images for acne analysis.
[0058] Step S1: In this embodiment, multispectral images obtained by a spectral skin image analyzer are selected. Based on the case of facial acne, parallel polarized light images and near-infrared spectral images are selected for acne identification.
[0059] Step S2: Identify the facial region and facial features of the parallel polarized light image. The method includes: using a first deep learning model (including but not limited to the YOLOv5 model) to identify the facial region and facial features of the parallel polarized light image, wherein the facial features include eyebrows, eyes, nose, mouth, ears, and hair.
[0060] This embodiment uses parallel polarized light images for facial feature extraction because parallel polarized light images are formed by light directly reflected from the skin surface, which is similar to RGB images taken by ordinary cameras, and thus more accurate for extracting facial feature regions.
[0061] Step S3: Construct an image mask based on the facial region and facial features. The method includes: annotating the facial region based on the facial features to obtain an image mask, wherein the non-acne growth region includes the eyebrows, eyes, mouth, ears, hair and nose area, and the acne growth region is the other areas in the facial region excluding the non-acne growth region.
[0062] Specifically, since acne does not occur all over the face, areas such as the mouth, eyes, eyebrows, hairline, and nose need to be excluded. Therefore, based on facial regions and features, this embodiment creates a face mask, as detailed below. Figure 2 Among them, the region with a mask value of 1 ( Figure 2 The white area represents the area where acne may occur, and the mask value is 2. Figure 2 The black area in the middle is the area where acne does not grow.
[0063] Step S4: Using the potential acne-growing areas in the image mask as a reference, binarize the near-infrared spectral image to obtain a near-infrared grayscale binary image. The method includes:
[0064] Convert the near-infrared spectral image into a near-infrared grayscale image;
[0065] Based on the potential acne-growing areas in the image mask, several pixels are uniformly sampled from left to right and from top to bottom on the near-infrared grayscale image at intervals of a preset number of pixels (10 pixels), and the brightness of each pixel is recorded.
[0066] The sampled pixels are sorted by brightness, and the median brightness is taken as the brightness threshold.
[0067] The brightness of each pixel in the near-infrared grayscale image is compared with a brightness threshold. Pixels below the brightness threshold are recorded as 1, and pixels above the brightness threshold are recorded as 0, thus generating a near-infrared grayscale binary image.
[0068] It is important to note that this embodiment chose near-infrared spectroscopy for acne identification because hemoglobin in the skin's blood vessels has a high absorption rate for near-infrared light with wavelengths of 780-1000 nm. Therefore, imaging the face using near-infrared spectroscopy can detect facial capillary dilation. Acne is always accompanied by inflammation, leading to capillary dilation. Therefore, near-infrared spectroscopy can effectively identify acne. Please refer to [link to relevant documentation]. Figure 3 ,in, Figure 3 (a) Acne as shown in a parallel polarized light image. Figure 3 (b) Acne as shown in a near-infrared spectral image. It is easy to see that the acne is clearer in the near-infrared spectral image.
[0069] Step S5: Perform acne identification on the near-infrared grayscale binary image, the method including:
[0070] In the near-infrared grayscale binary image, the pixels marked as 1 are considered as pixels that may have acne, and it is necessary to connect the scattered pixels to form acne areas;
[0071] The neighborhood algorithm is used to connect pixels that may cause acne to form several connected regions.
[0072] After generating connected regions, a high threshold T is set based on the number of pixels x in the connected regions. h With low threshold T l When the number of pixels x in the connected region is higher than the low threshold T l And below the high threshold T h If so, the connected region is considered to be acne;
[0073] The connected regions that may be acne are compared with the face mask. If more than a preset percentage (60%) of the pixels in the connected regions that may be acne are in the face mask, the connected regions are retained and considered as acne; otherwise, the connected regions are removed.
[0074] Step S6: Based on the first acne identification, a second identification is performed on the parallel polarized light image to remove the false identifications from the first identification. The method includes:
[0075] Based on the connected regions identified as acne in step S5, the bounding rectangles of these regions are cropped at the same location in the parallel polarized light image and used as training sample data. A second deep learning model (including but not limited to a ResNet50 model) is then trained using this training sample data. The trained second deep learning model is then used to perform a second acne identification on the parallel polarized light image to remove erroneous acne identified in step S5. Please refer to [link to relevant documentation]. Figure 4 ,in, Figure 4 (a) shows the results of the first acne identification. Figure 4 (b) shows the results of the second acne identification. It is not difficult to see that there were some misidentification results in the first acne identification. The second acne identification can effectively reduce the misidentification.
[0076] Example 2
[0077] This embodiment provides a facial acne recognition system, including:
[0078] Acquisition module: used to acquire multispectral images of a face, including parallel polarized light images and near-infrared spectral images;
[0079] First recognition module: used to recognize the facial region and facial features of the parallel polarized light image;
[0080] Construction module: used to construct an image mask based on the facial region and facial features, wherein the image mask includes regions where acne may grow and regions where acne does not grow;
[0081] Binarization module: Used to binarize the near-infrared spectral image based on the possible acne-growing areas in the image mask to obtain a near-infrared grayscale binary image;
[0082] The second recognition module is used to identify acne from the near-infrared grayscale binary image.
[0083] Example 3
[0084] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the facial acne recognition method described in Embodiment 1.
[0085] Example 4
[0086] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the facial acne recognition method described in Embodiment 1.
[0087] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.
[0088] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0089] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0090] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0091] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.
[0092] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.
Claims
1. A method for identifying facial acne, characterized in that: include: Step S1: Acquire a multispectral image of the face, the multispectral image including a parallel polarized light image and a near-infrared spectral image; Step S2: Identify the facial region and facial features in the parallel polarized light image; Step S3: Construct an image mask based on the facial region and facial features, wherein the image mask includes areas where acne may grow and areas where acne does not grow; Step S4: Using the potential acne-growing areas in the image mask as a reference, binarize the near-infrared spectral image to obtain a near-infrared grayscale binary image. The method includes: Convert the near-infrared spectral image into a near-infrared grayscale image; Based on the potential acne-growing areas in the image mask, several pixels are uniformly sampled at preset pixel intervals on the near-infrared grayscale image, and the brightness of each pixel is recorded. The sampled pixels are sorted by brightness, and the median brightness is taken as the brightness threshold. The brightness of each pixel in the near-infrared grayscale image is compared with a brightness threshold. Pixels below the brightness threshold are recorded as 1, and pixels above the brightness threshold are recorded as 0, thus generating a near-infrared grayscale binary image. Step S5: Perform acne identification on the near-infrared grayscale binary image, the method including: In the near-infrared grayscale binary image, pixels marked as 1 are considered as pixels that may have acne. The neighborhood algorithm is used to connect pixels that may cause acne to form several connected regions. After generating connected regions, based on the number of pixels in the connected regions Set high threshold With low threshold When the number of pixels in the connected region Higher than the low threshold And below the high threshold If so, the connected region is considered to be acne; The connected regions that may be acne are compared with the face mask. If more than a preset percentage of pixels in the connected regions that may be acne are in the face mask, the connected regions are retained and considered as acne; otherwise, the connected regions are removed.
2. The facial acne identification method according to claim 1, characterized in that: In step S2, the facial region and facial features of the parallel polarized light image are identified, and the method includes: The facial regions and facial features of the parallel polarized light image are identified using a first deep learning model, wherein the facial features include eyebrows, eyes, nose, mouth, ears, and hair.
3. The facial acne identification method according to claim 1, characterized in that: In step S3, an image mask is constructed based on the facial region and facial features. The method includes: The facial regions are labeled based on the facial features to obtain an image mask. The non-acne growth regions include the eyebrows, eyes, mouth, ears, hair, and nose wings. The acne-prone growth regions are the other areas of the face excluding the non-acne growth regions.
4. The facial acne identification method according to claim 1, characterized in that: The preset percentage is 60%.
5. The facial acne identification method according to claim 1, characterized in that: Following step S5, the method further includes: performing a second acne identification on the acne identification result of step S5 on the parallel polarized light image, the method comprising: Based on the connected regions identified as acne in step S5, the bounding rectangles of the connected regions identified as acne are cropped at the same position in the parallel polarized light image and used as training sample data. The training sample data is then used to train the second deep learning model. The trained second deep learning model is used to perform a second acne identification on the parallel polarized light image to remove the erroneous acne identified in step S5.
6. A facial acne recognition system for implementing the facial acne recognition method as described in any one of claims 1 to 5, characterized in that: include: Acquisition module: used to acquire multispectral images of a face, including parallel polarized light images and near-infrared spectral images; First recognition module: used to recognize the facial region and facial features of the parallel polarized light image; Construction module: used to construct an image mask based on the facial region and facial features, wherein the image mask includes regions where acne may grow and regions where acne does not grow; Binarization module: Used to binarize the near-infrared spectral image based on the possible acne-growing areas in the image mask to obtain a near-infrared grayscale binary image; The second recognition module is used to identify acne from the near-infrared grayscale binary image.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the facial acne recognition method as described in any one of claims 1 to 5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the facial acne recognition method as described in any one of claims 1 to 5.
Citation Information
Patent Citations
Acne-based relevance calculation method for human face and macaque face
CN117274673A
