Computing device, method and apparatus for predicting acne characteristics of keratin substances in human subjects

Keratin substance images were collected through microscope and machine learning models were used to extract feature data, which solved the problem of difficult to classify prone acne and predict acne frequency in the existing technology, achieved rapid and accurate acne prediction and classification, and simplified user operations .

CN118984939APending Publication Date: 2024-11-19LOREAL SA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202280094491.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2022-03-30
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The prior art is difficult to effectively classify keratin substances that are prone to acne and are not prone to acne, and it is difficult to accurately predict the frequency of acne in individuals. It depends on clinical experts and self-recording, and is inconvenient to use.

Method used

Using digital images collected based on microscopes, the characteristic data of the keratin substance region is extracted through machine learning models, and acne-prone information and acne frequency information are classified to provide fast and simple prediction results.

Benefits of technology

You can classify skin that is prone to acne without the knowledge of dermatologists. Users can obtain acne frequency information by themselves, simplifying the acne prediction process and improving the accuracy and convenience of prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118984939A_ABST
    Figure CN118984939A_ABST
Patent Text Reader

Abstract

The present disclosure relates to a computing device for predicting an acne characteristic of a keratin substance of a human subject, the computing device comprising: an acquisition unit comprising computing circuitry configured to acquire one or more digital images of a region of keratin substance of the human subject; an extraction unit comprising a calculation circuit configured to extract a plurality of feature data relating to acnes among all the pores from one or more digital images of the keratin material region of the human subject; a classification unit including a calculation circuit configured to classify the extracted feature data; and a display unit including a calculation circuit configured to display a result of the classification; wherein the classification result is easy-to-generate acne information or acne occurrence frequency information. The present disclosure also relates to a method, apparatus and computer readable medium for predicting acne characteristics of keratin substances of a human subject.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to a computing device, method and apparatus for predicting acne characteristics of keratinous materials of human subjects. More specifically, the present disclosure relates to predicting acne susceptibility information and acne frequency information. Background Art

[0002] Keeping keratin healthy is the most important thing. However, more and more people are suffering from keratin acne due to environmental, stress and keratin sensitivity. There are two problems for people. The first problem is that it is difficult to classify "keratin that is more prone to acne" and "keratin that is less prone to acne"; the second problem is that it is difficult to know the individual's "acne frequency history", especially recalling the frequency of acne history in any time period (for example, last month, last year, 2005, last February).

[0003] In order to solve these two problems, other AI skin diagnosis systems on the market, such as the Yue-Sai 4T system (Skinrun), have been developed based on normal-scale (non-microscopic) facial images, and a clinical routine evaluation method for distinguishing acne-prone skin has been developed, which is based on the evaluation of dermatologists through personal observation and combined with acne history examination. However, such current solutions also have some disadvantages, such as the need to combine the diagnosis of dermatologists and self-claimed acne history records to become acne-prone, and individuals are required to record the frequency of acne in great detail. Such current solutions rely on clinical experts, self-claimed acne history records, and acne frequency records, and are therefore not easily used by ordinary people.

[0004] Therefore, there is a need to predict acne susceptibility information and acne occurrence frequency information in a simple and effective manner. Summary of the invention

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0006] Various aspects and features of the disclosure are described in further detail below.

[0007] According to a first aspect of the present disclosure, there is provided a computing device for predicting acne characteristics of keratin substances of human subjects, the computing device comprising: an acquisition unit, the acquisition unit comprising a computing circuit configured to acquire one or more digital images of a keratin substance area of ​​the human subject; an extraction unit, the extraction unit comprising a computing circuit configured to extract a plurality of feature data related to acne in all pores from the one or more digital images of the keratin substance area of ​​the human subject; a classification unit, the classification unit comprising a computing circuit configured to classify the extracted feature data; and a display unit comprising a computing circuit configured to display a result of the classification; wherein the result of the classification is acne susceptibility information or acne occurrence frequency information.

[0008] In an embodiment of the first aspect, the computing device further comprises a data storage device comprising computing circuitry configured to store the digital image.

[0009] In an embodiment of the first aspect, the acquisition unit is selected from a microscope, and the microscope is preferably a reflective confocal microscope.

[0010] In an embodiment of the first aspect, the classification unit includes a computing circuit configured to classify a first group of feature data from the extracted feature data through a first machine learning model, so that the classification result is a result of acne susceptibility information.

[0011] In one embodiment of the first aspect, the classification unit includes a computing circuit configured to classify a second set of feature data from the extracted feature data through a second machine learning model, so that the classification result is a result of acne occurrence frequency information.

[0012] In an embodiment of the first aspect, the first machine learning model is trained by: obtaining clinical assessment data related to acne for multiple sampled human subjects; obtaining self-claimed data of acne history for the multiple sampled human subjects; acquiring one or more digital images of keratin material areas of the multiple sampled human subjects; extracting multiple feature data related to acne in all pores from the one or more digital images of the keratin material areas of the multiple sampled human subjects; and constructing the first machine learning model by using the clinical assessment data, the self-claimed data of acne history, and a first set of feature data from the extracted feature data.

[0013] In an embodiment of the first aspect, the second machine learning model is trained by: obtaining self-claimed data of acne history of the multiple sampled human subjects; collecting one or more digital images of keratin material areas of the multiple sampled human subjects; extracting multiple feature data related to acne in all pores from the one or more digital images of the keratin material areas of the multiple sampled human subjects; and constructing the second machine learning model by using the self-claimed data of acne history and a second set of feature data from the extracted feature data.

[0014] In an embodiment of the first aspect, the first set of feature data includes a ratio of hyperkeratotic hair follicles.

[0015] In an embodiment of the first aspect, the second characteristic data includes a ratio of hyperkeratinized follicles, a ratio of follicles with thick keratinized edges, and a ratio of follicles with internal keratin content.

[0016] In one embodiment of the first aspect, the ratio of hyperkeratotic follicles is obtained by manually counting hyperkeratotic follicles among all hair follicles or by an automatic segmentation algorithm.

[0017] According to a second aspect of the present disclosure, a method for predicting acne characteristics of keratin material of a human subject is provided, the method comprising: acquiring one or more digital images of a keratin material area of ​​the human subject; extracting a plurality of feature data related to acne in all pores from the one or more digital images of the keratin material area of ​​the human subject; classifying the extracted feature data; and displaying a result of the classification; wherein the result of the classification is acne susceptibility information or acne occurrence frequency information.

[0018] In an embodiment of the second aspect, the method further comprises storing the digital image.

[0019] In an embodiment of the second aspect, said acquiring one or more digital images of the keratinous material area of ​​the human subject is performed by a microscope, preferably a reflective confocal microscope.

[0020] In an embodiment of the second aspect, the classifying the extracted feature data further comprises classifying a first group of feature data from the extracted feature data through a first machine learning model, so that the classification result is a result of acne susceptibility information.

[0021] In an embodiment of the second aspect, classifying the extracted feature data further comprises classifying a second group of feature data from the extracted feature data through a second machine learning model, so that the classification result obtained is a result of acne occurrence frequency information.

[0022] In an embodiment of the second aspect, the first machine learning model is trained by: obtaining clinical assessment data related to acne for multiple sampled human subjects; obtaining self-claimed data of acne history for the multiple sampled human subjects; collecting one or more digital images of keratin material areas of the multiple sampled human subjects; extracting multiple feature data related to acne in all pores from the one or more digital images of the keratin material areas of the multiple sampled human subjects; and constructing the first machine learning model by using the clinical assessment data, the self-claimed data of acne history, and a first set of feature data from the extracted feature data.

[0023] In an embodiment of the second aspect, the second machine learning model is trained by: obtaining self-claimed data of acne history of the multiple sampled human subjects; collecting one or more digital images of keratin material areas of the multiple sampled human subjects; extracting multiple feature data related to acne in all pores from the one or more digital images of the keratin material areas of the multiple sampled human subjects; and constructing the second machine learning model by using the self-claimed data of acne history and a second set of feature data from the extracted feature data.

[0024] In an embodiment of the second aspect, the first set of feature data includes a ratio of hyperkeratotic hair follicles.

[0025] In an embodiment of the second aspect, the second feature data includes a ratio of hyperkeratinized follicles, a ratio of follicles with thick keratinized edges, and a ratio of follicles with internal keratin content.

[0026] In an embodiment of the second aspect, the ratio of hyperkeratotic follicles is obtained by manually counting hyperkeratotic follicles among all hair follicles or by an automatic segmentation algorithm.

[0027] According to a third aspect of the present disclosure, a computer-readable medium is provided, on which instructions are stored. When the instructions are executed, a computing device is prompted to perform the above-mentioned method.

[0028] According to a fourth aspect of the present disclosure, there is provided an apparatus for predicting acne properties of keratinous material of a human subject, the apparatus comprising means for performing the above mentioned method.

[0029] According to the disclosure, the solution of the present invention greatly improves existing solutions, and any user can classify acne-prone skin using only imaging methods without the need for the expertise of a dermatologist. In addition, any user can classify their personal recent history of acne frequency without knowing or requiring them to record and track it themselves. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] By way of example, the above and other aspects, features and benefits of various embodiments of the present disclosure will become more fully apparent from the following detailed description with reference to the accompanying drawings, in which the same reference numerals or letters are used to designate the same or equivalent elements. The accompanying drawings are illustrated to facilitate a better understanding of the embodiments of the present disclosure and are not necessarily drawn to scale, in which:

[0031] Figure 1 A block diagram illustrating a computing device according to a first aspect of the present disclosure;

[0032] FIG2( a ) illustrates a screenshot of an overview image of a hair follicle having acne-prone keratinous material;

[0033] FIG2( b ) illustrates a screenshot of a partial image of a hair follicle having acne-prone keratinous material;

[0034] Figure 3 A flow chart illustrating a method of predicting acne properties of keratinous material in a human subject according to a second aspect of the present disclosure;

[0035] Figure 4 A flow chart illustrating a method for predicting acne susceptibility information of keratinous material of a human subject according to the second aspect of the present disclosure;

[0036] Figure 5 A flowchart illustrating construction of a first machine learning model according to the present disclosure is shown;

[0037] Figure 6 An example of constructing the first machine learning model by using a decision tree is shown;

[0038] Figure 7 A flowchart illustrating construction of a second machine learning model according to the present disclosure; and

[0039] Figure 8 An example of constructing the second machine learning model by using a decision tree is shown. DETAILED DESCRIPTION

[0040] Embodiments herein will be described in detail below with reference to the accompanying drawings, in which embodiments are shown. However, these embodiments herein may be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. The elements of the accompanying drawings are not necessarily proportional to each other. The same numbers always refer to the same elements.

[0041] The terms used in this article are only for the purpose of describing specific embodiments and are not intended to be limiting. As used in this article, the singular forms "a", "an", and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It will be further understood that when the terms "comprise", "comprises", "includes", and / or "comprising" are used in this article, the existence of the features, integers, steps, operations, elements and / or components set forth is specified, but the existence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof is not excluded.

[0042] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood. It will be further understood that the terms used herein should be interpreted as having a meaning consistent with their meaning in the context of this specification and the relevant art, and unless so clearly defined in this article, the terms used herein should not be interpreted as an idealized or overly formal meaning.

[0043] The present technology is described below with reference to the block diagrams and / or flowchart illustrations of the methods, devices (systems) and / or computer program products according to the present embodiments. It is to be understood that the blocks in the block diagrams and / or flowchart illustrations and the combination of the blocks in the block diagrams and / or flowchart illustrations can be implemented by computer program instructions. These computer program instructions can be provided to a processor, controller or control unit of a general-purpose computer, a special-purpose computer and / or other programmable data processing device to produce a machine so that instructions executed by a processor of a computer and / or other programmable data processing device create a device for implementing the functions / actions specified in one or more block diagrams and / or flowchart blocks.

[0044] Thus, the present technology may be embodied in hardware and / or software (including firmware, resident software, microcode, etc.). Furthermore, the present technology may take the form of a computer program product on a computer-usable or computer-readable storage medium having computer-usable or computer-readable program code embodied in the medium for use by or in connection with an instruction execution system. In the context of this document, a computer-usable or computer-readable medium may be any medium that may contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0045] Embodiments herein will be described below with reference to the accompanying drawings.

[0046] The inventive concept of the present disclosure is that the solution of the present invention is based on an imaging device / tool ​​to observe and segment the hyperkeratinization signals of the skin pores of a consumer's healthy-looking skin, and then identify whether they have "more acne-prone" or "less acne-prone" skin, and identify their acne occurrence frequency via a pre-built machine learning prediction model.

[0047] Therefore, the present disclosure is directed to identifying whether people have "acne-prone" skin and / or whether they are in an acne-prone skin population based on imaging acquisition (e.g., confocal microscopy), and also to classifying the "acne occurrence frequency" of people based on imaging acquisition (e.g., confocal microscopy).

[0048] Figure 1 A block diagram of a computing device 100 according to a first aspect of the present disclosure is illustrated. The computing device 100 comprises an acquisition unit 101, an extraction unit 102, a classification unit 103 and a display unit 104, which can communicate with each other via a wireless or wired connection. The acquisition unit 101 acquires one or more digital images of a keratinous material area of ​​a human subject. Such an acquisition unit 101 can be a microscope, such as it is preferably a reflective confocal microscope.

[0049] Microscope (Reflection confocal microscopy (RCM) can be a solution validated by clinical studies) can show 3D imaging of skin pores / follicles on the facial area. Principle of RCM: It is a non-invasive imaging technology that has been used to identify keratin signals and their distribution within the epidermis, which measures the structure of the sublayers of the skin with cellular resolution. "Excessive keratin around hair follicles" provides a strong endogenous contrast due to the mismatch of refractive index with other components in skin tissue. Normal hair follicles are round black "pores" and should not have a high density of keratin signals, while keratinized pores, which are early symptoms of acne, will further develop into acne. It is further proved that the trace keratin symptoms around hair follicles on skin areas that appear healthy to consumers can be directly linked to their acne susceptibility.

[0050] The acquisition unit 101 includes a computing circuit for obtaining an image of the epidermis level of the facial area. In a preferred embodiment, the computing device 100 also includes a data storage device for storing the image. The extraction unit 102 can extract feature data, such as excessive keratinization signals of multiple hair follicles / skin pores. The classification unit 103 includes a computing circuit configured to classify the extracted feature data. The display 104 can be any display including a computing circuit configured to display one or more instances of predicted acne data of keratin substances on a graphical user interface. The prediction result can be acne susceptibility information or acne occurrence frequency information.

[0051] The computing device 100 can be, for example, a server of a service provider, a device associated with a client (e.g., a client device), a system on a chip, and / or any other suitable computing device or computing system. In various embodiments, the computing device 100 can take a variety of different configurations. For example, the computing device 100 can be implemented as a computer-like device, including a personal computer, a desktop computer, a multi-screen computer, a laptop computer, a netbook, etc. The computing device 100 can also be implemented as a device similar to a mobile device, including mobile devices such as mobile phones, portable music players, portable game devices, tablet computers, multi-screen computers, etc. The computing device 100 can also be implemented as a TV-like device, including a device having or connected to a generally larger screen in a casual viewing environment. These devices include televisions, set-top boxes, game consoles, etc.

[0052] In one embodiment, the computing circuit includes, among other things, one or more computing devices, such as a processor (e.g., a microprocessor, a quantum processor, a qubit processor, etc.), a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., or any combination thereof, and may include discrete digital or analog circuit elements or electronic devices, or combinations thereof. In one embodiment, the computing circuit includes one or more ASICs having a plurality of predefined logic components. In one embodiment, the computing circuit includes one or more FPGAs, each of which has a plurality of programmable logic components.

[0053] In one embodiment, the computing circuit includes one or more electronic circuits, printed circuits, flexible circuits, electrical conductors, electrodes, cavity resonators, conductive traces, ceramic patterned electrodes, electromechanical components, transducers, and the like.

[0054] In one embodiment, the computing circuit includes one or more components that are operably coupled to each other (e.g., communicatively, electromagnetically, magnetically, ultrasonically, optically, inductively, electrically, capacitively coupled, wirelessly coupled, etc.). In one embodiment, the circuit includes one or more remotely located components. In one embodiment, the remotely located components are operably coupled, e.g., via wireless communication. In one embodiment, the remotely located components are operably coupled, e.g., via one or more communication modules, receivers, transmitters, transceivers, etc.

[0055] In one embodiment, the computing circuit includes a memory, such as one for storing instructions or information. Non-limiting examples of memory include volatile memory (e.g., random access memory (RAM), dynamic random access memory (DRAM), etc.), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM), etc.), permanent memory, etc. Further non-limiting examples of memory include erasable programmable read-only memory (EPROM), flash memory, etc. In one embodiment, the memory is coupled to, for example, one or more computing devices via one or more instruction, information, or power buses. In one embodiment, the computing circuit includes one or more databases stored in the memory. In one embodiment, the computing circuit includes one or more lookup tables stored in the memory.

[0056] In one embodiment, the computing circuitry includes one or more computer readable media drives, interface sockets, universal serial bus (USB) ports, memory card slots, etc., and one or more input / output components, such as, for example, a graphical user interface, a display, a keyboard, a keypad, a trackball, a joystick, a touch screen, a mouse, switches, dials, etc., and any other peripheral devices. In one embodiment, the computing circuitry includes one or more user input / output components operably coupled to at least one computing device configured to control (electrically, electromechanically, software-implemented, firmware-implemented, or other control, or a combination thereof) at least one parameter associated with determining one or more tissue thermal characteristics, for example, in response to a detected shift in the on-voltage.

[0057] In one embodiment, the computing circuitry includes: electrical circuits forming a general-purpose computing device configured by a computer program (e.g., a general-purpose computer configured by a computer program that at least partially executes the processes and / or devices described herein, or a microprocessor configured by a computer program that at least partially executes the processes and / or devices described herein); electrical circuits forming a memory device (e.g., in the form of memory (e.g., random access, flash, read-only, etc.)); electrical circuits forming a communication device (e.g., a modem, a communication switch, an optoelectronic device, etc.); and / or any non-electrical analog thereof, such as an optical or other analog.

[0058] The computing device 100 according to the first aspect of the present disclosure can quickly provide acne information or acne occurrence frequency, and deliver quick results and consistent classification results.

[0059] In order to classify the acne proneness information or the acne occurrence frequency information, two machine learning models are pre-established. During the establishment of the first machine learning model and the second machine learning model, multiple parameters are loaded. Table 1 illustrates a table of example parameters loaded in machine learning. As shown in Table 1, parameters such as percentage of hyperkeratosis, percentage of thickening, percentage of irregularity, percentage of inclusions, average diameter of the top 3 of the stratum corneum hair follicles, etc. are obtained from the microscope. However, the model does not select other parameters, which means that they are not key factors. It was found that only the parameters from the microscope were the most important for classification.

[0060]

[0061] Table 1

[0062] FIG. 2( a) illustrates a screenshot of an overview image of a hair follicle with keratinous material that is prone to acne. As shown in FIG. 2( a), by such an acquisition unit, a digital image with hair follicles can be obtained, and the acquisition unit can be a microscope, preferably a reflective confocal microscope. FIG. 2( b) illustrates a screenshot of a local image of a hair follicle with keratinous material that is prone to acne. Hair follicles with hyperkeratinization, hair follicles with thick keratinized edges, irregularly shaped hair follicles, and hair follicles with internal keratin content are shown in FIG. 2( b). From FIG. 2( b), it is also found that compared with skin that is less prone to acne, keratinous material that is prone to acne has a higher ratio of hyperkeratinized hair follicles, a higher ratio of thick keratinized edges, more irregularly shaped hair follicles, and more hair follicles with internal keratin content. It is found that one or more of such parameters are most important for the classification of acne susceptibility information and acne occurrence frequency information.

[0063] For the machine learning model, the same parameters (such as the raw data example of Table 1) are loaded into the machine learning model training for two different models, and it is found that the automatically selected parameters for the two different models are different. In one embodiment, for the first machine learning used to classify acne-prone information, the key parameter is the ratio of hyperkeratotic follicles. In one embodiment, for the second machine learning used to classify acne occurrence frequency information, the key parameters are the ratio of hyperkeratotic follicles, the ratio of follicles with thickened keratinized edges, and the ratio of follicles with internal keratin content. Of course, these parameters are just examples, and the present disclosure is not limited to such parameters. Such automatically selected parameters can vary.

[0064] Figure 3 A flow chart of a method 300 for predicting acne characteristics of keratinous materials of a human subject according to the second aspect of the present disclosure is illustrated. At step 301 of the method 300, one or more digital images of a keratinous material area of ​​a human subject are collected by an imaging acquisition device. Such an imaging acquisition device can be a device such as a reflective confocal microscope, which includes a computing circuit for obtaining an image of the epidermis level of a facial area. Then, as shown at step 302, feature data, such as excessive keratinization signals of multiple hair follicles / skin pores, are extracted from the image. Then, at step 303, the extracted feature data is classified. Finally, at step 304, one or more instances of the classified acne data of the keratinous material are displayed on a graphical user interface. The result of the classification can be acne susceptibility information or acne occurrence frequency information.

[0065] Figure 4 A flow chart 400 illustrating a method for predicting acne susceptibility information of keratinous material of a human subject according to the second aspect of the present disclosure. Figure 4 As shown in , an image of the epidermis level of the facial area is obtained by an imaging acquisition device / tool ​​(such as a reflective confocal microscope), as shown at step 401. Then, at step 402, the hyperkeratinization signals of multiple hair follicles / skin pores are extracted. The inventors have found that the signal of hyperkeratinized pores on the facial area of ​​a person is highly correlated with the clinical classification of "acne-prone skin" and the person's acne history. Based on this, the inventors developed a first machine learning model that classifies keratin substance types as "more acne-prone" or "less acne-prone" skin based on the obtained images.

[0066] For any new consumer, we will need them to perform image acquisition, and our system can give a prediction result of keratin substances about whether he / she is "more prone to acne" or "less prone to acne" based on the parameter "ratio of hyperkeratinized hair follicles" via the first machine learning model, as shown at step 403. Finally, such prediction results about acne-prone information are displayed, as shown at step 404. Of course, the input parameters for classifying acne-prone information are only examples, and the present disclosure is not limited to such parameters.

[0067] Compared to the classic clinical assessment of "acne-prone" classification, which requires face-to-face observation by a dermatologist expert and the consumer in combination with the consumer's self-declared acne history to draw a conclusion, the present invention is an AI-based automated system that delivers fast and consistent classification results that can be easily used even by non-clinical experts. It can be a branded over-the-counter device or a laboratory device / system.

[0068] In addition to classifying keratin types as "more acne-prone" or "less acne-prone," Figure 3 The technical solution of the method 300 in the embodiment can also classify the "acne occurrence frequency" based on some input parameters via a second machine learning model as described in detail below. In one embodiment, such input parameters for classifying the "acne occurrence frequency" include the ratio of hyperkeratinized follicles, the ratio of follicles with thick keratinized edges, and the ratio of follicles with internal keratin content. Of course, these parameters are only examples, and the present disclosure is not limited to such parameters.

[0069] Figure 5 Flowchart 500 for constructing a first machine learning model according to the present disclosure is illustrated. As mentioned above, the first machine learning model is pre-trained to classify keratin substances that are "more prone to acne" or keratin substances that are "less prone to acne". During the establishment of the first machine learning model, clinical assessment data related to acne for a plurality of sampled human subjects is obtained at step 501. Such clinical assessment can be a dermatologist's diagnosis of "keratin substances that are more prone to acne" or "keratin substances that are less prone to acne". Then, at step 502, self-claimed data of acne history of the plurality of sampled human subjects is obtained. For example, the self-claimed acne history can be the average state related to acne in the past 1 year. Through such clinical assessment data and self-claimed acne history, criteria for "acne-prone skin = 1" and "less prone to acne skin = 0" can be specified. Then, at step 503, one or more digital images of skin areas of the plurality of sampled human subjects are acquired. In one example, the digital image can be an image at the epidermal level of a facial area.

[0070] Then, at step 504, a plurality of feature data related to acne in all pores are extracted from the one or more digital images of the keratinous material area of ​​the plurality of sampled human subjects.

[0071] Finally, at step 505, the first machine learning model is constructed by using the clinical assessment data, the self-claimed data of acne history, and a first set of feature data from the extracted feature data. In one example, the parameter "ratio of hyperkeratinized hair follicles" is extracted as the first set of feature data, because it has been shown that the signal of hyperkeratinized pores on the facial area of ​​a person is highly correlated with the clinical classification of "acne-prone skin". In one example, the first machine learning model for classifying "acne-prone" is constructed using the data from steps 501, 502, 504 using the "scikit-learn" package from OpenCV [1].

[0072] For a clinical trial (N=15 subjects), the test showed that the size of the hair follicles in the acne-prone skin range was larger than the size of the hair follicles in the less acne-prone skin. It was also found that the acne-prone keratinous material had a higher ratio of hyperkeratinized hair follicles, a higher ratio of thick keratinized rims, more irregularly shaped hair follicles, and more hair follicles with internal keratin content compared to the less acne-prone skin, as shown in Figure 2(b). The RCM measurement results also showed a high correlation with the clinical symptoms of inflammatory acne, PIE (post-inflammatory erythema), PIH (post-inflammatory hyperpigmentation), and the frequency of self-reported history of acne. It was also found that the ratio of hyperkeratinized hair follicles and thick keratin rims decreased after 4 weeks of treatment with the facial serum product.

[0073] Figure 6 The diagram shows an example of constructing the first machine learning model by using a decision tree. Figure 6 As shown in , it is a machine learning model of Model 1-acne-prone skin and less acne-prone skin. In one example, by using "hyperkeratinization percentage" as input (INPUT), the output (OUTPUT) is "acne-prone skin" or "keratin material that is less prone to acne". In one example, the predetermined value for the hyperkeratinization percentage is set to 40.9%, and the predetermined value can vary according to actual needs. If the hyperkeratinization percentage>40.9%, the classification = "acne-prone keratin material" is the right branch. If the hyperkeratinization percentage <=40.9%, the classification = "less acne-prone skin" is the left branch.

[0074] For new users, epidermal-level skin images of facial areas with multiple hair follicles / skin pores hyperkeratotic signals are obtained by using an imaging acquisition device / tool ​​(e.g., reflectance confocal microscope). The user can then obtain VIP parameters (e.g., "keratinized follicle ratio") for acne prone classification from the images by processing the images either by manually counting hyperkeratotic follicles among all follicles (e.g., 45% of a total of 30 follicles are keratinized) or by an automated segmentation algorithm to obtain the above parameters. The results from the VIP parameters are input into a Figure 6 Example of a Decision Tree shown in , where the output is the classification of "Acne Prone".

[0075] Through such a first machine learning model, acne-prone skin can be rapidly predicted and fast and consistent classification results can be delivered.

[0076] Figure 7 The flowchart of constructing the second machine learning model according to the present disclosure is illustrated. As mentioned above, the second machine learning model is pre-trained to classify "acne occurrence frequency". During the second machine learning model, at step 701, self-claimed data of acne history of multiple sampled human subjects are obtained. For example, the self-claimed acne history can be the average state related to acne in the past year. Then, at step 702, one or more digital images of the keratin material area of ​​the multiple sampled human subjects are collected. In one example, the digital image can be an epidermal-level image of the facial area.

[0077] Then, at step 703, a plurality of feature data related to acne in all pores are extracted from the one or more digital images of the keratinous material area of ​​the plurality of sampled human subjects.

[0078] Finally, at step 704, the second machine learning model is constructed by using the self-claimed data of acne history and the second set of feature data from the extracted feature data. In one example, the parameters "ratio of hyperkeratinized follicles", "ratio of follicles with thick keratinized edges" and "ratio of follicles with internal keratin content" are extracted as the second set of feature data, because it has been proven that the "ratio of hyperkeratinized follicles", "ratio of follicles with thick keratinized edges" and "ratio of follicles with internal keratin content" are highly correlated with the clinical classification of "acne occurrence frequency level". However, the second set of feature data is not limited to the parameters "ratio of hyperkeratinized follicles", "ratio of follicles with thick keratinized edges" and "ratio of follicles with internal keratin content", and can be changed according to actual needs. In one example, the second machine learning model for classifying "acne occurrence frequency" is constructed using the "scikit-learn" package from OpenCV[1] using the data from steps 701 and 703.

[0079] For a clinical trial (N=15 subjects), the individual self-declared acne frequency level can be 0-3 (N=15). It was demonstrated that the individual self-declared acne frequency level had a high correlation with the instrumental results in "hyperkeratosis" (0.67) and "percentage of thick keratinized edges" (0.717).

[0080] Figure 8 The figure shows an example of constructing the second machine learning model by using a decision tree. Figure 8 As shown in, it is a model 2-acne occurrence frequency classification machine learning model, by using "ratio of hyperkeratinized follicles", "ratio of follicles with thick keratinized edges" and "ratio of follicles with internal keratin content" as input (INPUT), the output (OUTPUT) is CLASS (class) = 0, CLASS = 1, CLASS = 2, where Class 0 = "acne occurs less than once every 3 months", Class 1 = "acne occurs once a month"; Class 2 = "acne occurs every two weeks". The predetermined values ​​corresponding to "ratio of hyperkeratinized follicles", "ratio of follicles with thick keratinized edges" and "ratio of follicles with internal keratin content" are set to 39.7%, 23.6% and 17.2%, respectively. It is noted that the corresponding predetermined values ​​corresponding to "ratio of hyperkeratinized follicles", "ratio of follicles with thick keratinized edges" and "ratio of follicles with internal keratin content" are not limited to the above values, and can be changed according to actual needs.

[0081] In one example, if hyperkeratosis <= 39.7%, then the occurrence frequency level = 0. If hyperkeratosis > 39.7% and thick cornified rim <= 23.6%, then the occurrence frequency level = 2. If hyperkeratosis > 39.7%, thick cornified rim > 23.6% and internal keratin content > 17.2%, then the occurrence frequency level = 2. If hyperkeratosis > 39.7%, thick cornified rim < 23.6% and internal keratin content <= 17.2%, then the occurrence frequency level = 1.

[0082] For new users, an epidermal-level skin image of a facial area with multiple hair follicles / skin pores hyperkeratinization signals is obtained by using an imaging acquisition device / tool ​​(e.g., a reflectance confocal microscope). Then, from the image, the user can obtain parameters for acne occurrence frequency classification, "ratio of hyperkeratinized follicles", "ratio of follicles with thick keratinized rims", and "ratio of follicles with internal keratin content", where the ratio of hyperkeratinized follicles can be obtained by either manually counting hyperkeratinized follicles among all follicles (e.g., 45% of a total of 30 follicles are keratinized) or by processing the image with an automatic segmentation algorithm. The obtained parameters are input into a Figure 8 Example of a Decision Tree shown in Figure 1, where the output is a classification of "Frequency of Acne".

[0083] Through such a second machine learning model, acne frequency information can be quickly predicted, and fast and consistent classification results can be delivered.

[0084] The method for implementing the first machine learning model and the second machine learning model of the present disclosure is not limited to the decision tree method. Instead, they can also be established using a deep learning method.

[0085] By using a technological solution, the results are shown to the consumer, letting them know their "acne prone category" and "acne frequency category". The invention can be applied anywhere where there is no need for an on-site dermatologist, such as over the counter, and can work at any time, immediately after image acquisition or some time later. Our inventive device can be a branded over the counter device for consumers to get results before / after applying makeup. It can also be a personalized tool to guide the consumer in choosing the right product for his skin condition.

[0086] An embodiment of the present disclosure may be an article of manufacture in which instructions (e.g., computer code) are stored on a non-transitory machine-readable medium (such as a microelectronic memory) that program one or more data processing components (collectively referred to herein as "processors") to perform the operations described above. In other embodiments, some of these operations may be performed by specific hardware components that include hard-wired logic (e.g., dedicated digital filter blocks and state machines). Alternatively, these operations may be performed by any combination of programmed data processing components and fixed hard-wired circuit components.

[0087] Although embodiments have been illustrated and described herein, it will be appreciated by those skilled in the art that various changes and modifications may be made, and that equivalents may be substituted for elements thereof, without departing from the true scope of the present technology. In addition, many modifications may be made to accommodate specific circumstances and the teachings herein without departing from the central scope thereof. Therefore, it is intended that the present embodiments are not limited to the specific embodiments disclosed as the best mode contemplated for carrying out the present technology, but that the present embodiments include all embodiments falling within the scope of the appended claims.

Claims

1. A computing device for predicting acne properties of keratinous material of a human subject, the computing device comprising: an acquisition unit comprising computing circuitry configured to acquire one or more digital images of a region of keratinous material of said human subject; an extraction unit comprising a computing circuit configured to extract a plurality of feature data associated with acne in all pores from one or more digital images of the keratinous material area of ​​the human subject; a classification unit, comprising a computing circuit configured to classify the extracted feature data; as well as a display unit including a computing circuit configured to display a result of the classification; The classification result is acne susceptibility information or acne occurrence frequency information.

2. The computing device according to claim 1, wherein: The computing device also includes a data storage device for storing the digital image.

3. The computing device according to claim 1, wherein: The acquisition unit is embodied by a microscope, and the microscope is preferably a reflective confocal microscope.

4. The computing device according to claim 1, wherein: The classification unit includes a computing circuit configured to classify a first group of feature data from the extracted feature data through a first machine learning model, so as to obtain a result of the classification being a result of acne susceptibility information.

5. The computing device according to claim 1, wherein: The classification unit includes a computing circuit configured to classify a second set of feature data from the extracted feature data through a second machine learning model, so that the classification result is a result of acne occurrence frequency information.

6. The computing device according to claim 4, wherein: The first machine learning model is trained by the following steps: obtaining clinical assessment data related to acne for a plurality of sampled human subjects; obtaining self-reported data on the medical history of acne from the plurality of sampled human subjects; acquiring one or more digital images of keratinous material areas of the plurality of sampled human subjects; extracting a plurality of feature data related to acne in all pores from the one or more digital images of the keratinous material area of ​​the plurality of sampled human subjects; as well as The first machine learning model is constructed by using the clinical assessment data, the self-claimed data of acne medical history, and a first set of feature data from the extracted feature data.

7. The computing device according to claim 5, wherein: The second machine learning model is trained by the following steps: obtaining self-reported data on the medical history of acne from the plurality of sampled human subjects; acquiring one or more digital images of keratinous material areas of the plurality of sampled human subjects; extracting a plurality of feature data related to acne in all pores from the one or more digital images of the keratinous material area of ​​the plurality of sampled human subjects; as well as The second machine learning model is constructed by using the self-claimed data of acne medical history and a second set of feature data from the extracted feature data.

8. The computing device according to claim 4 , wherein: The first set of characteristic data includes a ratio of hyperkeratotic hair follicles.

9. The computing device according to claim 5 , wherein: The second characteristic data includes a ratio of hyperkeratotic follicles, a ratio of follicles with thick keratinized edges, and a ratio of follicles with internal keratin content.

10. The computing device according to claim 8 , wherein: The ratio of hyperkeratotic follicles was obtained by manually counting hyperkeratotic follicles among all follicles or by an automatic segmentation algorithm.

11. A method for predicting acne properties of keratinous material in a human subject, the method comprising: acquiring one or more digital images of a region of keratinous material of the human subject; extracting a plurality of feature data associated with acne in all pores from one or more digital images of the keratinous material area of ​​the human subject; Classifying the extracted feature data; as well as displaying the results of said classification; The classification result is acne susceptibility information or acne occurrence frequency information.

12. The method of claim 11, further comprising storing the digital image.

13. The method according to claim 11, wherein: The acquiring of one or more digital images of the keratinous material area of ​​the human subject is performed by a microscope, preferably a reflective confocal microscope.

14. The method according to claim 11, wherein: Classifying the extracted feature data also includes classifying a first group of feature data from the extracted feature data through a first machine learning model, so that the classification result is a result of acne susceptibility information.

15. The method according to claim 11, wherein: Classifying the extracted feature data also includes classifying a second group of feature data from the extracted feature data through a second machine learning model, so that the classification result is a result of acne occurrence frequency information.

16. The method according to claim 14, wherein: The first machine learning model is trained by the following steps: obtaining clinical assessment data related to acne for a plurality of sampled human subjects; obtaining self-reported data on the medical history of acne from the plurality of sampled human subjects; acquiring one or more digital images of keratinous material areas of the plurality of sampled human subjects; extracting a plurality of feature data related to acne in all pores from the one or more digital images of the keratinous material area of ​​the plurality of sampled human subjects; as well as The first machine learning model is constructed by using the clinical assessment data, the self-claimed data of acne medical history, and a first set of feature data from the extracted feature data.

17. The method according to claim 15, wherein: The second machine learning model is trained by the following steps: obtaining self-reported data on the medical history of acne from the plurality of sampled human subjects; acquiring one or more digital images of keratinous material areas of the plurality of sampled human subjects; extracting a plurality of feature data related to acne in all pores from the one or more digital images of the keratinous material area of ​​the plurality of sampled human subjects; as well as The second machine learning model is constructed by using the self-claimed data of acne medical history and a second set of feature data from the extracted feature data.

18. The method according to one of claims 14 and 16, wherein: The first set of characteristic data includes a ratio of hyperkeratotic hair follicles.

19. The method according to one of claims 15 and 17, wherein: The second characteristic data includes a ratio of hyperkeratotic follicles, a ratio of follicles with thick keratinized edges, and a ratio of follicles with internal keratin content.

20. The method according to one of claims 18 and 19, wherein: The ratio of hyperkeratotic follicles was obtained by manually counting hyperkeratotic follicles among all follicles or by an automatic segmentation algorithm.

21. A computer readable medium having stored thereon instructions which, when executed, cause a computing device to perform the method according to any one of claims 11-20.

22. An apparatus for predicting acne properties of keratinous material of a human subject, the apparatus comprising means for performing the method according to one of claims 11-20.