Intelligent dandruff detection system and method

Abstract

The present invention discloses an intelligent dandruff detection system and method, wherein the intelligent dandruff detection system includes an operation module, a first neural network module, a second neural network module, and a classification module. The operation module is used to receive a scalp area image of a subject and convert the scalp area image into a first feature map. The first neural network module is electrically connected to the operation module and is used to receive a scalp area image. The first neural network module adopts a convolutional neural network model and converts the scalp area image into a second feature map. The second neural network module is electrically connected to the first neural network module and is used to receive the second feature map. The second neural network module adopts a self-attention network model and converts the second feature map into a third feature map. The classification module is electrically connected to the second neural network module, and is used to receive the third feature map and output a score to measure the severity of the subject's dandruff.

Description

Technical Field

[0001] The present invention relates to a scalp detection system and method, and in particular to an intelligent dandruff detection system and method. Background Art

[0002] Dandruff is a problem that almost everyone has encountered. Some people with mild dandruff can solve the problem by buying anti-dandruff shampoo on the market. However, for some people with more serious dandruff, it may be necessary to seek professional help. However, since the relevant staff need a period of professional training to be competent in judging the dandruff situation, this will increase the relevant costs.

[0003] Therefore, how to quickly and inexpensively detect the severity of a subject's dandruff is something that is worth considering by those with ordinary knowledge in the art. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent dandruff detection system and method, which can automatically determine the severity of the dandruff of the subject, thereby reducing the labor cost.

[0005] The intelligent dandruff detection system of the present invention is used to detect the severity of a subject's dandruff problem, and includes an operation module, a first neural network module, a second neural network module, and a classification module. The operation module is used to receive a scalp area image of the subject and convert the scalp area image into a first feature map. The first neural network module is electrically connected to the operation module and is used to receive the scalp area image. The first neural network module adopts a convolutional neural network model and converts the scalp area image into a second feature map. The second neural network module is electrically connected to the first neural network module and is used to receive the second feature map. The second neural network module adopts a self-attention network model and converts the second feature map into a third feature map. The classification module is electrically connected to the second neural network module, and the classification module is used to receive the third feature map and output a score. The above score is used to measure the severity of the subject's dandruff.

[0006] The intelligent dandruff detection system as described above further includes a scalp detector, which is used to photograph an area of ​​the subject's scalp to form a scalp area image. Moreover, the scalp detector is, for example, a digital microscope.

[0007] For example, in the above-mentioned intelligent dandruff detection system, during the training phase, the intelligent dandruff detection system is trained using a semi-supervised learning method.

[0008] The intelligent dandruff detection system as mentioned above further includes a compression module, which is connected between the first neural network module and the second neural network module, wherein the compression module receives and compresses the second feature map and transmits the compressed second feature map to the second neural network module.

[0009] As in the above-mentioned intelligent dandruff detection system, the operation module is installed on at least one terminal device, and the first neural network module, the second neural network module, and the classification module are installed on at least one server.

[0010] The smart dandruff detection method of the present invention is used to detect the severity of dandruff of a subject, and the smart dandruff detection method includes the following steps. First, obtain at least one scalp area image of the subject. Then, input the scalp area image into a convolutional neural network model, and output a second feature map. Next, input the second feature map into a self-attention network model, and output a third feature map. Then, classify according to the third feature map and output a score. The score is used to measure the severity of the subject's dandruff.

[0011] In the above-mentioned intelligent dandruff detection method, a scalp detector is used to photograph an area of ​​the scalp of the subject to form a scalp area image.

[0012] The smart dandruff detection method as described above further includes the following steps: compressing the second feature map and transmitting the second feature map to the self-attention network model.

[0013] In order to make the above features and advantages of the present invention more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Various embodiments will be described below with reference to the accompanying drawings, which are intended to be illustrative and not intended to be limiting in any way, wherein like reference numerals represent like components, and wherein:

[0015] Figure 1 FIG. 1 is a schematic diagram of an embodiment of the intelligent dandruff detection system of the present invention.

[0016] Figure 2 Depicted is a schematic diagram of a user interface of an operation module.

[0017] Figure 3 Shown is a human scalp region segmentation diagram.

[0018] Figure 4 The illustrated embodiment is an example of the intelligent dandruff detection method of the present invention.

[0019] Figure 5 and Figure 6 Shown is the distribution of dandruff on the scalp.

[0020] Figure 7 FIG. 1 is a schematic diagram of the information flow in the intelligent dandruff detection system of the present invention.

[0021] Figure 8 The actual dandruff condition corresponding to various scores is shown.

[0022] Fig. 9 FIG. 1 is a structural diagram of another embodiment of the intelligent dandruff detection system of the present invention. DETAILED DESCRIPTION

[0023] The present invention is best understood with reference to the detailed description and accompanying drawings set forth herein. Various embodiments will be discussed below with reference to the accompanying drawings. However, it will be readily appreciated by those skilled in the art that the detailed description given herein with respect to the accompanying drawings is for illustrative purposes only, as these methods and systems may extend beyond the described embodiments. For example, the teachings given and the needs of a particular application may produce a variety of optional and suitable methods to implement the functions of any of the details described herein. Therefore, any method may extend beyond the specific implementation selection range in the following embodiments described and shown.

[0024] Certain words are used in the specification and subsequent patent applications to refer to specific components. It should be understood by those with ordinary knowledge in the field that hardware manufacturers may use different terms to refer to the same component. This specification and subsequent patent applications do not use differences in names as a way to distinguish components, but rather use differences in the functions of components as the criteria for distinction. The word "including" mentioned throughout the specification and subsequent claims is an open-ended term and should be interpreted as "including but not limited to". In addition, the word "coupled" here includes any direct and indirect electrical connection means. Therefore, if the text describes a first device coupled to a second device, it means that the first device can be directly electrically connected to the second device, or indirectly electrically connected to the second device through other devices or connection means.

[0025] See also Figure 1 , Figure 1Illustrated is an architecture diagram of an embodiment of the intelligent dandruff detection system of the present invention. In this embodiment, the intelligent dandruff detection system 100 includes an operation module 110, a first neural network module 120, a second neural network module 130, and a classification module 140. The operation module 110 is, for example, disposed in a terminal device 20, and the terminal device 20 is, for example, a desktop computer, a laptop computer, a tablet computer, or a smart phone. In addition, the first neural network module 120, the second neural network module 130, and the classification module 140 are disposed on at least one server 10, that is, the cloud, and the server 10 and the terminal device 20 can be connected via a network. In this embodiment, the operation module 110 provides a user interface 112 (such as Figure 2 The terminal device 20 is connected to a scalp detector 150, and the staff can use the scalp detector 150 to photograph a certain area of ​​the scalp of the subject. In this embodiment, the scalp detector 150 is, for example, a digital microscope, and the digital microscope is, for example, a digital microscope DMC-1213 produced by VAST Technologies INC. Of course, those with ordinary knowledge in the art can also select other types of digital microscopes as embodiments of the scalp detector 150.

[0026] Next, please also refer to Figure 1 , Figure 4 ,and Figure 7 , Figure 4 The following is an example of the smart dandruff detection method of the present invention. Figure 7 The diagram is a schematic diagram of the information flow in the intelligent dandruff detection system of the present invention. First, step S110 is executed to obtain at least one scalp area image of the subject. In this embodiment, 8 scalp area images of the subject are captured, and the acquisition method is described as follows.

[0027] Please also refer to Figure 2 and Figure 3 , Figure 2 The diagram shown is a schematic diagram of the user interface of the operation module. Figure 3 The figure shows the segmentation of the human scalp region. Figure 3 In , the human scalp is divided into 8 areas, marked as area A, area B, area C, area D, area E, area F, area G, and area H. Figure 3The scalp segmentation areas shown are based on the paper "Assessing therapeutic effectiveness of scalp treatments for dandruff and seborrheic dermatitis, part 1: a reliable and relevant method based on the adherent scalp flaking score (ASFS)" (hereinafter referred to as ASFS). However, those with ordinary knowledge in the field can also use other methods to segment the human scalp, not limited to Figure 3 When the staff uses the scalp detector 150 to photograph the scalp of the subject, the photographed scalp image will be displayed in real time in the scalp display area 1122 of the user interface 112 (such as Figure 2 After adjusting the angle of the camera, the staff will measure each area of ​​the subject's scalp (i.e. Figure 3 The photo is taken in the scalp image area 1124 .

[0028] Afterwards, re-refer to Figure 1 and Figure 4, implement step S120, and transmit the scalp area image from the operation module 100 to the first neural network module 120. In this embodiment, the operation module 100 first converts the scalp area images of the eight regions into vectors (hereinafter referred to as the first feature map), and then transmits them to the first neural network module 120 through the network. In this embodiment, the first neural network module 120 adopts a convolutional neural network model, and the convolutional neural network model is mainly composed of a convolution layer. In the convolution layer of the convolutional neural network model, a filter is used to filter the features, so the required features can be strengthened and the unnecessary features can be weakened or excluded. For example, the subject's scalp may not only have dandruff, but also other features, such as hair, redness, wounds, or spots caused by bacterial infection. However, the function of the intelligent dandruff detection system 100 is mainly to determine the severity of the subject's dandruff, so the dandruff feature can be strengthened by the filter and other scalp features of the subject can be weakened. In this embodiment, the convolutional neural network model is a normalizer-free deep residual network model (Normalizer-FreeResNets), which is derived from the paper "High-Performance Large-Scale Image Recognition Without Normalization". Of course, those with general knowledge in this field can also use other types of convolutional neural network models, such as: VGGNet, ResNet, EfficientNet, or MobileNet.

[0029] In addition, in some embodiments, the first neural network module 120 may also include a pooling layer, the main purpose of which is to reduce the number of parameters to avoid overfitting and to achieve an anti-noise effect. In addition, since the data output by the first neural network module 120 (hereinafter referred to as the second feature map) is directly output to the second neural network module 130, unlike a general convolutional neural network, the first neural network module 120 of this embodiment does not have a fully connected layer.

[0030] Next, step S130 is implemented to input the second feature map into the second neural network module 130 connected to the first neural network module 120. The second neural network module 130 adopts the self-attention network (Transformer networks) model, which was proposed by Google in the paper Attention Is All You Need in 2017. As for why the self-attention network model is adopted by the second neural network module 130, and what difference will be caused if the second neural network module 130 adopts the convolutional neural network model, it will be described in detail below.

[0031] Please refer to Figure 5 and Figure 6 , Figure 5 and Figure 6 The diagram shows the distribution of dandruff on the scalp. Figure 5 The distribution of dandruff 30 is relatively limited to a certain area, while in Figure 6 The dandruff 30 is scattered over a wider area. Due to its own characteristics, the convolutional neural network model can only perceive local features, and may not be able to consider the correlation between two distant areas. In other words, the receptive field of the convolutional neural network model is small. Therefore, for Figure 5 and Figure 6 These two types of dandruff distribution, even if Figure 6 The amount of dandruff 30 Figure 5 There are more, the convolutional neural network model may still determine Figure 5 On the contrary, the self-attention network model uses the self-attention mechanism, so it can take into account the correlation between two distant regions. In other words, the receptive field of the self-attention network model is larger, so it can more accurately judge when the distribution of dandruff is more dispersed (such as Figure 6 shown).

[0032] In this embodiment, the self-attention network model adopts a visual self-attention network (Vision Transformer networks) model, which is derived from the paper "AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE". Of course, those with general knowledge in this field can also adopt other types of self-attention network models, such as CaiT, LV-ViT, or BEiT.

[0033] After the second feature map is converted into the third feature map by the second neural network module 130, step S140 is performed to output the third feature map to the classification module 140. In this embodiment, the classification module 140 is composed of two fully connected layers. As shown in step S150, after receiving the third feature map and performing calculations, the classification module 140 outputs a score. In this embodiment, the scoring basis is also shown in the ASFS paper, which is divided into the following six conditions:

[0034] 1) Zero points: no dandruff;

[0035] 2) Two points: fine dandruff (slight desquamation, in small pieces);

[0036] 3) Four points: flaky dandruff (small degree of scaly scalp, in flakes);

[0037] 4) Six points: blocky dandruff (moderate dandruff, in blocks);

[0038] 5) 8 points: clumping dandruff (moderate degree of dandruff, in clumping form);

[0039] 6) Very severe: dandruff in clumps (the dandruff is thick and in clumps).

[0040] Also, please refer to Figure 8 As shown, it shows the actual dandruff conditions corresponding to various scores. Of course, the above-mentioned scoring criteria are only one embodiment, and those with general knowledge in the field can also use other scoring criteria. After the classification module 140 outputs the score, the score is transmitted to the operation module 110 and displayed on the user interface 112. In this embodiment, the intelligent dandruff detection system 100 scores the eight scalp areas of the subject respectively, but those with general knowledge in the field should understand that the entire scalp of the subject can also be scored in a total or weighted manner. After obtaining the score of the subject's dandruff problem, the staff can propose an improvement plan that meets the subject's needs based on the score to improve the subject's dandruff problem.

[0041] Please also refer to Fig. 9 , Fig. 9The diagram is a structural diagram of another embodiment of the intelligent dandruff detection system of the present invention. Compared with the intelligent dandruff detection system 100, the intelligent dandruff detection system 200 of this embodiment has an additional compression module 160, which is connected between the first neural network module 120 and the second neural network module 130. After receiving the second feature map from the first neural network module 120, the compression module 160 compresses the second feature map and transmits the compressed second feature map to the second neural network module 130. In other words, after the convolutional neural network model outputs the second feature map, the second feature map is compressed and transmitted to the self-attention network model.

[0042] In the present embodiment, the compression module 160 compresses the second feature map by using a convolution operation. For example, the second feature map before compression is a three-dimensional array of [N×N×M], where M is greater than 3. Afterwards, a convolution operation is performed on the three-dimensional array of [N×N×M] and a three-dimensional array of [1×1×3] to obtain a three-dimensional array of [N×N×3], which is the compressed second feature map. By the above method, unnecessary computational complexity can be reduced, and the first neural network module 120 and the second neural network module 130 can be effectively connected in series to achieve a more stable recognition effect.

[0043] In this embodiment, the intelligent dandruff detection system 100 is trained using semi-supervised learning during the training phase. This is because the number of professionals in scalp detection is small and their salaries are high, so if supervised learning is used, the labeling cost will increase significantly. In detail, this embodiment uses a fixed matching (FixMatch) algorithm to implement semi-supervised learning. The fixed matching algorithm is derived from the paper "FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence". Of course, the above description does not mean that the present invention can only be trained using semi-supervised learning. Those with general knowledge in this field can also use supervised learning or other types of methods to train the intelligent dandruff detection system 100.

[0044] In addition, after each terminal device 20 uploads the scalp area images of the subject, these uploaded scalp area images can also be used for further training of the intelligent dandruff detection system 100, so that the intelligent dandruff detection system 100 becomes more and more accurate.

[0045] In summary, the intelligent dandruff detection system 100 disclosed in this case has the following advantages:

[0046] (1) Using artificial intelligence to score the scalp of the subject, the scalp testing staff can do the job even if they have not received rigorous professional training, thus reducing costs;

[0047] (2) Combine the convolutional neural network model with the self-attention network model to increase the accuracy of dandruff scoring;

[0048] (3) The operation module is set in the terminal device, while the part responsible for artificial intelligence calculation, namely: the first neural network module, the second neural network module, and the classification module, are set in the cloud. In this way, when the artificial intelligence calculation part is updated, all terminal devices can benefit from the latest update.

[0049] The above embodiments are merely examples for the convenience of explanation. Although any modifications are made by those skilled in the art, they will not deviate from the scope of protection as set forth in the claims.

Claims

1. An intelligent dandruff detection system for detecting the severity of a subject's dandruff problem, the intelligent dandruff detection system comprising: An operating module, configured to receive a plurality of scalp region images from different preset scalp regions of the subject, and convert each scalp region image into a corresponding first feature map; a first neural network module, electrically connected to the operation module, configured to receive the first feature map and convert the first feature map into a second feature map, wherein the first neural network module adopts a non-normalized deep residual convolutional neural network model; a compression module, electrically connected to the first neural network module, configured to receive and compress the second feature map; a second neural network module, electrically connected to the compression module, configured to receive the compressed second feature map and convert the second feature map into a third feature map, wherein the second neural network module adopts a visual self-attention network model; and a classification module, electrically connected to the second neural network module, configured to receive the third feature map and output a score; Among them, this score is used to measure the severity of the subject's dandruff.

2. The intelligent dandruff detection system according to claim 1 further comprises a scalp detector, which is used to photograph an area of ​​the subject's scalp to form an image of the scalp area.

3. The intelligent dandruff detection system according to claim 2, wherein the scalp detector is a digital microscope. 4 . The intelligent dandruff detection system according to claim 1 , wherein during the training phase, the intelligent dandruff detection system is trained using a semi-supervised learning method.

5. The intelligent dandruff detection system according to claim 1, wherein the scoring is based on the following six conditions: Zero points: no dandruff; Two points: fine dandruff; Four points: flaky dandruff; Six points: blocky dandruff; Eight points: clumpy dandruff; Very: clumpy dandruff.

6. The intelligent dandruff detection system according to claim 1, wherein the operation module is installed on at least one terminal device, and the first neural network module, the second neural network module, and the classification module are installed on at least one server.

7. An intelligent dandruff detection method for detecting the severity of dandruff of a subject, the intelligent dandruff detection method comprising: (a) obtaining a plurality of scalp area images of the subject, each scalp area image corresponding to a predetermined scalp area; (b) inputting the scalp area image into a non-normalized deep residual convolutional neural network model, wherein the non-normalized deep residual convolutional neural network model is configured to convert the scalp area image into a second feature map; (c) compressing the second feature map; (d) inputting the compressed second feature map into a visual self-attention network model to generate and output a third feature map; and (e) performing classification according to the third feature map and outputting a score; Among them, this score is used to measure the severity of the subject's dandruff.

8. The intelligent dandruff detection method according to claim 7, further comprising the following steps before step (a): A scalp detector is used to photograph a plurality of scalp regions of the subject's scalp to form an image of the scalp region.

9. The intelligent dandruff detection method according to claim 7, wherein the scoring is based on the following six conditions: Zero points: no dandruff; Two points: fine dandruff; Four points: flaky dandruff; Six points: blocky dandruff; Eight points: clumpy dandruff; Very: clumpy dandruff.

Images