Processing method and device based on scalp image data

Abstract

The embodiment of the invention relates to a processing method and device based on scalp image data. The method comprises the following steps: acquiring first sample data of the scalp image data; a pre-training scalp classification model is constructed based on a SpectFormer model; training, verifying and testing the pre-trained scalp classification model through the first sample data to generate a scalp classification model; inputting target type image data in the first sample data into a scalp classification model, and performing scalp classification output; generating second sample data according to a scalp classification output result; constructing a pre-training target type data evaluation model based on a YOLOv8 model; training, verifying and testing the pre-trained target type data evaluation model through the image data marked with the target type evaluation sign in the second sample data to generate a target type data evaluation model; wherein the second sample data is constructed by adopting an active learning strategy.

Description

Technical Field

[0001] The present invention relates to the field of digital medical technology, and in particular to a processing method and device based on scalp image data. Background Art

[0002] Currently, artificial intelligence is increasingly used in skin recognition. In terms of scalp image classification, it is mainly based on traditional deep learning models. However, due to the high-resolution characteristics of scalp images and the discreteness, subtlety and similarity of key classification signs, the image information density is high. Traditional deep learning models cannot effectively capture and retain key signs. Therefore, when evaluating the scalp based on scalp image data, only a small number of scalps can be evaluated by binary classification. At the same time, the recognition results of most intelligent models lack interpretability and can only directly provide the final output, and cannot effectively explain their decision-making process. This lack of transparency may cause us to ignore the correct results obtained accidentally due to errors in the reasoning process, thereby limiting the improvement of the model.

[0003] How to classify various scalps for scalp images, how to effectively identify and locate key features after classification, and how to perform explainable visual displays have become urgent problems that need to be solved. Summary of the invention

[0004] The purpose of the present invention is to address the defects of the prior art and provide a processing method and device based on scalp image data, to classify various scalps for scalp images and to effectively identify and locate target type evaluation signs from target type image data after classification.

[0005] To achieve the above object, the present invention provides a first aspect of a processing method based on scalp image data, comprising:

[0006] Acquire first sample data of scalp image data, wherein the first sample data includes a plurality of scalp types;

[0007] Building a pre-trained scalp classification model based on the SpectFormer model; training, verifying and testing the pre-trained scalp classification model using the first sample data to generate a scalp classification model;

[0008] Inputting the target type image data in the first sample data into the scalp classification model, and performing scalp classification output, wherein the scalp classification output includes the true label of the image and the possible category predicted by the model and the corresponding accuracy rate;

[0009] Generate second sample data according to the result output by the scalp classification;

[0010] A pre-trained target type data evaluation model is constructed based on the YOLOv8 model; the pre-trained target type data evaluation model is trained, verified and tested through image data with target type evaluation signs marked in the second sample data to generate a target type data evaluation model; wherein the second sample data is constructed using an active learning strategy.

[0011] Furthermore, after the pre-trained target type data evaluation model is trained, verified and tested by the image data with target type evaluation signs marked in the second sample data and the target type data evaluation model is generated, the method further includes:

[0012] Inputting the scalp image data to be detected into the scalp classification model, and outputting the scalp classification;

[0013] If the scalp is classified as a target type, the scalp image data to be detected is input into the target type data evaluation model, and a target type evaluation sign is output; according to the target type evaluation sign, it is determined whether the target type is in an active or inactive period, and an evaluation report is generated.

[0014] Furthermore, after inputting the scalp image data to be detected into the scalp classification model and outputting the scalp classification, the method further includes:

[0015] The output of the scalp classification is visually interpreted using gradient-weighted class activation maps.

[0016] Furthermore, the target type is alopecia areata, and the target type assessment signs include black dot sign, yellow dot sign, exclamation mark hair, broken hair, short vellus hair, and branching blood vessels.

[0017] Further, the step of determining whether the scalp is in an active or inactive period according to the target type evaluation sign is specifically as follows:

[0018] If one or more of the target type assessment signs of black spot sign, exclamation mark hair, broken hair and short vellus hair are present, the scalp is in the active stage.

[0019] Furthermore, the multiple scalp types also include: androgenic alopecia, cicatricial alopecia, trichotillomania, psoriasis, seborrheic dermatitis, pigmented nevus, seborrheic keratosis, viral warts and non-scalp healthy skin.

[0020] Further, generating second sample data according to the result of the scalp classification output specifically includes:

[0021] Calculating the entropy of the accuracy rate of each target type image data after passing through the scalp classification model, the possible types are alopecia areata, androgenic alopecia and cicatricial alopecia;

[0022] Arrange multiple target type image data from large to small according to the calculated entropy.

[0023] Further, the pre-trained scalp classification model is trained, verified and tested by the first sample data to generate a scalp classification model, specifically including:

[0024] Training the pre-trained scalp classification model using the training set in the first sample data to generate a scalp classification baseline model, wherein the resolution of the scalp image data in the training set is 224×224;

[0025] The baseline weights and biases are interpolated to the scalp classification baseline model adapted to the 512×512 resolution through the Bicubic interpolation method;

[0026] Freeze the parameters of the baseline and unfreeze the parameters related to bias, class embedding, normalization, and scaling factors;

[0027] The scalp classification baseline model is trained, verified and tested by converting the first sample data into a 512×512 resolution to generate a scalp classification model.

[0028] A second aspect of the present invention provides a processing device based on scalp image data, comprising:

[0029] A first data construction module, used to obtain first sample data of scalp image data, wherein the first sample data includes multiple scalp types;

[0030] A classification model building module, used to build a pre-trained scalp classification model based on the SpectFormer model; train, verify and test the pre-trained scalp classification model through the first sample data to generate a scalp classification model;

[0031] A scalp classification output module, used to input the target type image data in the first sample data into the scalp classification model and perform scalp classification output, wherein the scalp classification output includes the real label of the image and the possible category predicted by the model and the corresponding accuracy rate;

[0032] A second data construction module, used to generate second sample data according to the result output by the scalp classification;

[0033] An evaluation model construction module is used to construct a pre-trained target type data evaluation model based on the YOLOv8 model; the pre-trained target type data evaluation model is trained, verified and tested through image data with target type evaluation signs marked in the second sample data to generate a target type data evaluation model; wherein the second sample data is constructed using an active learning strategy.

[0034] Furthermore, the device also includes:

[0035] A result visualization module is used to visualize the output of the scalp classification using gradient weighted class activation mapping.

[0036] An embodiment of the present invention provides a processing method and device based on scalp image data, which builds a scalp classification model based on the SpectFormer model, performs multiple scalp classifications on the scalp image data, and builds a target type data evaluation model based on the YOLOv8 model, which effectively identifies and locates target type evaluation signs from the classified target type image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 One of the flowcharts of the processing method based on scalp image data provided in the first embodiment of the present invention;

[0038] Figure 2 A scalp classification model framework diagram of the scalp image data processing method provided in the first embodiment of the present invention;

[0039] Figure 3 A second flowchart of the processing method based on scalp image data provided in the first embodiment of the present invention;

[0040] Figure 4 A third flowchart of the processing method based on scalp image data provided in the first embodiment of the present invention;

[0041] Figure 5 A model classification heat map of the scalp image data processing method provided in the first embodiment of the present invention;

[0042] Figure 6 This is a schematic diagram of the structure of a processing device based on scalp image data provided in the second embodiment of the present invention;

[0043] Figure 7 A second structural diagram of a processing device based on scalp image data provided in the second embodiment of the present invention;

[0044] Figure 8 This is a third structural diagram of a scalp image data processing device provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments.

[0047] The processing method and device based on scalp image data provided by the embodiment of the present invention build a scalp classification model based on the SpectFormer model, and classify various scalps for scalp image data; build a target type data evaluation model based on the YOLOv8 model, and effectively identify and locate target type evaluation signs from the classified target type image data; adopt an active learning strategy to quickly build a training set used for the target type data evaluation model; after freezing the parameters of the baseline by using the Bicubic interpolation method, selectively unfreeze the relevant specific parameters, and adjust the pre-trained scalp classification model to a high-resolution model in a low-cost manner, thereby achieving fast fine-tuning while minimizing the interference with the pre-training weights; and use gradient-weighted class activation mapping to achieve a visual interpretation of the output of scalp classification, and automatically generate a scalp image data evaluation report.

[0048] Embodiment 1

[0049] Figure 1 This is a flowchart of a processing method based on scalp image data provided in the first embodiment of the present invention. Figure 1 , the technical solution of the present invention is described with specific embodiments.

[0050] Step 110: Acquire first sample data of scalp image data, where the first sample data includes multiple scalp types.

[0051] Specifically, the first sample data may be derived from a dermoscopic image database, mainly including the Fitzpatrick skin type III or IV, and the scalp image data are all from the scalp part and a part of the normal dermoscopic image database of the non-scalp part for comparison. In a possible implementation, the multiple scalp types include: alopecia areata, androgenic alopecia, cicatricial alopecia (including localized scleroderma, lichen planus and discoid lupus erythematosus), trichotillomania, psoriasis, seborrheic dermatitis, pigmented nevus, seborrheic keratosis, viral warts and non-scalp healthy skin.

[0052] Step 120, constructing a pre-trained scalp classification model based on the SpectFormer model; training, verifying and testing the pre-trained scalp classification model using the first sample data to generate a scalp classification model.

[0053] Specifically, Figure 2 This is a scalp classification model framework diagram of the scalp image data processing method provided in the first embodiment of the present invention, such as Figure 2As shown in the figure, the model is based on the SpectFormer model framework, which combines the frequency domain layer and the self-attention layer. SpectFormer is a new visual Transformer model. The scalp image data is first linearly divided and positionally embedded. The frequency domain layer extracts local frequency domain features. The self-attention layer further extracts deep features. Finally, the features are output through the classification head to output the classification results.

[0054] Among them, the self-attention layer uses the same structure as De i T (Data-efficient Image Transformers), which is composed of a conventional normalization layer, a multi-head self-attention layer (MHSA) and a multi-layer perceptron (MLP) stack.

[0055] Among them, the frequency domain layer passes through a frequency domain gating network W c The network consists of a Fast Fourier Transform (FFT), a weighted gating layer and an Inverse Fast Fourier Transform (IFFT). It examines the input scalp image data from a complex perspective, captures the different frequency components of the scalp image data to extract the local frequency domain features. FFT converts the image in the spatial domain to the frequency domain, filters the different frequency components through the weighted gating layer, and IFFT converts the frequency domain back to the spatial domain to maintain the consistency of the input and output of each module. The learnable weight parameters of the weighted gating layer can be updated by back propagation during the model training process. In image processing, low-frequency components and high-frequency components refer to the components of image signals in different frequency ranges. Low-frequency components usually refer to the slowly changing parts of an image, which contain the structural data and shape information of the image. In the case of scalp image data, they refer to skin lesion data, such as skin lesion color data, skin lesion shape data, overall hair distribution data, etc. High-frequency components represent the rapidly changing parts of an image, which contain the details and texture information of the image. In the case of scalp image data, high-frequency components involve more subtle manifestations of skin lesions, such as hair follicle opening data, vascular pattern data, single hair morphology data, etc. These data features are used to identify subtle differences and early changes in the scalp.

[0056] In a possible implementation manner, step 120 specifically includes:

[0057] Before training, the first sample data is preprocessed and enhanced. First, the high-resolution scalp image data is randomly cropped within a preset range of aspect ratios, and the cropped image is scaled to a preset pixel value. The optional preset range is (0.75, 1.3333). The scaling uses bicubic interpolation to ensure the quality of scaling. The scaled image is randomly flipped horizontally to improve the symmetry recognition ability of the model. All images are rotated and color adjusted by the RandAugment data enhancement method to increase sample diversity. The processed image is converted into a tensor format to adapt to the deep learning framework. The pixel values ​​are then standardized and adjusted using the mean and standard deviation of the ImageNet dataset to stabilize the training process. The above data preprocessing and data enhancement process not only expands the scale of the dataset, but also improves the model's recognition of lesion features.

[0058] In a possible implementation manner, step 120 specifically further includes:

[0059] The learning rate of the scalp classification model uses a cosine annealing learning rate scheduler. First, use 20% of the data in the first sample data to find a suitable learning rate through the l earn.l r_find() function, and finally set the initial learning rate to 0.001 to ensure that the model can quickly adapt to the data in the early stage of training; then introduce learning rate noise to increase the exploration ability of the model, and the percentage limit of the learning rate noise is set to 0.67, that is, the learning rate will be affected by the noise for about 67% of the time during the training process. The standard deviation of the noise is set to 1.0 to limit the size of the learning rate change; then set the warm-up learning rate, gradually increasing from 1 / 2 of the initial learning rate—5e-4 to 1e-3, so that the model can better adapt to training; finally set the minimum value of the learning rate to 1e-4 to prevent division by zero errors. Through the adjustment of the above parameters, the scalp classification model can increase the performance of the model while ensuring the training speed.

[0060] In a possible implementation manner, step 120 specifically further includes steps A1-A4:

[0061] Step A1: train the pre-trained scalp classification model using the training set in the first sample data to generate a scalp classification baseline model, wherein the resolution of the scalp image data in the training set is 224×224.

[0062] In step A2, the baseline weights and biases are interpolated to a scalp classification baseline model adapted to a 512×512 resolution by a Bi cub ic interpolation method.

[0063] In step A3, the parameters of the baseline are frozen, and the parameters related to the bias, class embedding, normalization, and scale factor are unfrozen. Here, compared with the method of fine-tuning the entire model in the existing model, the method of adjusting only the relevant parameters accounts for less than 0.3% of the parameters, which minimizes the interference with the pre-trained weights while achieving fast fine-tuning.

[0064] Step A4, training, verifying and testing the scalp classification baseline model by converting the first sample data into a 512×512 resolution to generate a scalp classification model.

[0065] In a specific example, scalp image data with a resolution of 224×224 is fed into a pre-trained scalp classification model for training for 2000 rounds, and the converged model is used as the scalp classification baseline model; the baseline weights, biases, etc. are interpolated to the scalp classification baseline model adapted to the 512×512 resolution through Bi cub ic interpolation, the baseline parameters are frozen, and the parameters related to the bias, class embedding, normalization, and scale factor are unfrozen; the first sample data with a resolution of 512×512 containing more information is further trained for 300 rounds, and it is captured that the accuracy on the validation set in the first sample data converges from 56.886% to more than 87% in only 20 rounds, exceeding the baseline accuracy, and then the convergence trend slows down, reaching the best accuracy of 93.114% in about 140 rounds, and the best accuracy is achieved on both the training set and the validation set in the first sample data. The performance of the scalp classification model on the classification task in the validation set of the first sample data is shown in the following table:

[0066]

[0067] Among them, F1 is the harmonic mean of the precision and recall rate of the scalp classification model, which is used to measure the performance of the model while maintaining a balance between precision and recall rate. The value range of F1 is 0 to 1, where 1 represents the best performance and 0 represents the worst performance. In the test, the scalp classification model showed an excellent level in the validation set of the first sample data, with an overall accuracy of 0.931, a sensitivity and specificity of 0.931 and 0.992 respectively, an accuracy of 0.941, and an F1 score of 0.931.

[0068] Step 130: input the target type image data in the first sample data into the scalp classification model, and perform scalp classification output, wherein the scalp classification output includes the true label of the image and the possible category predicted by the model and the corresponding accuracy rate.

[0069] Among them, the possible scalp categories predicted by the model include alopecia areata, androgenic alopecia, cicatricial alopecia, trichotillomania, psoriasis, seborrheic dermatitis, pigmented nevus, seborrheic keratosis, viral warts and non-scalp healthy skin, a total of 10 categories.

[0070] Step 140: Generate second sample data according to the scalp classification output result.

[0071] In a possible implementation, when the target type is alopecia areata, step 140 includes steps B1 and B2:

[0072] Step B1, calculate the entropy of the accuracy of the possible types of alopecia areata, androgenic alopecia and scarring alopecia after each target type image data passes through the scalp classification model. Among them, the larger the entropy, the more information the image contains. When the image data of the target type of alopecia areata passes through the scalp classification model, the accuracy of the possible types of alopecia areata, androgenic alopecia and scarring alopecia outputted is the main one, and the accuracy of the other 7 categories is usually less than 1%, which can be ignored.

[0073] Step B2: Arrange the plurality of target type image data from large to small according to the calculated entropy.

[0074] Step 150, constructing a pre-trained target type data evaluation model based on the YOLOv8 model; training, verifying and testing the pre-trained target type data evaluation model through image data with target type evaluation signs labeled in the second sample data to generate a target type data evaluation model; wherein the second sample data is constructed using an active learning strategy.

[0075] Specifically, when the target type is alopecia areata, the target type assessment signs include black dot sign, yellow dot sign, exclamation mark hair, broken hair, short vellus hair, branching blood vessels, etc.

[0076] In combination with steps B1 and B2, in a possible implementation, step 150 includes steps C1-C3:

[0077] Step C1, select the first preset number of target type image data from the sorted multiple target type image data to perform target type assessment sign annotation, and use them as the first round of training sets for pre-training the target type data assessment model. In a specific example, the first preset number is 200, and the first 200 samples that best represent the target type scalp classification are selected from the sorted target type image data to perform target type assessment sign annotation as the training set, which is used for the first round of training of the pre-trained target type assessment model.

[0078] Step C2, select the first second preset number of target type image data after the first preset number from the sorted multiple target type image data, and input them into the pre-trained target type data evaluation model that has undergone the first round of training, annotate and correct the output image data with target type evaluation signs, update the training set, and train the pre-trained target type data evaluation model for the second round. In a specific example, the second preset number is 400, select the 201st to 600th images of the sorted target type image data and input them into the pre-trained target type data evaluation model that has undergone the first round of training, perform standard correction on the output 400 images with target type evaluation signs, and update the training set as the second round of training set, which includes the first 600 images annotated with target type evaluation signs.

[0079] Step C3, looping step C2 on the remaining second sample data until all the second sample data are labeled.

[0080] In the process of labeling all the second sample data, the active learning strategy is used to extract the most representative data first, which can improve the reasoning effect of the trained model at the beginning, thereby saving time during the next labeling correction. Because the key features on the scalp image are very dense, the image annotations can be as many as 60 to 100, and the average number of features per image is about 29.11.

[0081] In an alternative approach, Figure 3 The second flowchart of the processing method based on scalp image data provided in the first embodiment of the present invention is as follows: Figure 3 As shown, after step 150, the following steps are further included:

[0082] Step 210: input the scalp image data to be detected into the scalp classification model, and output the scalp classification.

[0083] Step 220: If the scalp is classified as a target type, the scalp image data to be detected is input into a target type data evaluation model, and a target type evaluation sign is output; according to the target type evaluation sign, it is determined whether the target type is in an active or inactive period, and an evaluation report is generated.

[0084] Among them, the target type evaluation sign results can be obtained by calling the built-in visualization method of YOLOv8, drawing a rectangular box on the original image to represent the prediction result, and using different colors and sign labels to represent different prediction signs.

[0085] Specifically, if the output scalp classification is a non-target type, only the classification result is output; if it is a target type, further target type evaluation sign detection is performed; when the target type is alopecia areata, if there is one or more target type evaluation signs of black spot sign, exclamation mark hair, broken hair and short vellus hair, it is determined that the target type is in an active period, otherwise it is in an inactive period, and an evaluation report is generated.

[0086] In an alternative approach, Figure 4 The third flowchart of the processing method based on scalp image data provided in the first embodiment of the present invention is as follows: Figure 4 As shown, after step 210, the following steps are also included:

[0087] Step 310 , visually interpret the output of scalp classification using gradient weighted class activation mapping.

[0088] Specifically, in computer vision tasks, Gradient-weighted Class Activation Mapping (Grad-CAM) is an algorithm used for visualization, which can provide an intuitive explanation for the important dependence of the final output of the neural network on the original image. By calculating the gradient weight of each position on the feature map, a heat map is generated, which can show the most important area in the input image for the model decision.

[0089] In the scalp classification model, the output weight composition that is effective for calculating the visualized thermal distribution includes batch size, downsampling of the original features and conversion to one dimension and number of channels after training and feature extraction. The existing visual Transformer visualization output weight composition also includes category labels, but category labels used in scalp classification models will lead to incorrect thermal imaging. Specifically, based on the SpectFormer scalp classification model, the input image is first divided into 16×16 uniform patches of equal size. If the input image uses 512×512 pixels, there are 32×32 patches to ensure that every spatial position of the image is covered and does not overlap. The input (1×1024×384) of the first normalization layer of the last SpectFormer module of the scalp classification model is selected as the target layer to calculate the gradient. For the category of the network output, the gradient of the category relative to the 1024 patches of the target layer is calculated. These gradients represent the contribution of each patch of the target layer to the output category. The contribution of the channel to the output category is obtained by performing global average pooling on the gradient of each channel (384). A 1×1024 patch matrix of weighted average gradient is converted into a two-dimensional heat map using the reshape function, which is the importance of each patch. Different threshold intervals are used as the most concerned position, which can be mapped to red. Other positions far away from the threshold area are mapped to different colors according to the distance. The position far away from the concerned area is set to colorless. The selection interval of the RGB classification threshold can be randomly generated first. For example, the threshold interval is 0~1 and a point is taken every 0.1, for a total of 45 cases. Then, based on the empirical value, it is confirmed whether the randomly selected interval is an optimal threshold for input training. The parameters include the starting point and the end point. Here, a label is not assigned to each interval, but 20×45=900 cases are constructed using 20 images under different threshold conditions, and the implementation effect of each image is judged. A simple multi-layer perceptron is trained using the two variables of threshold interval and implementation effect. The final threshold is selected as 0.1~0.3 and presented as the final mapping. Optionally, the corresponding areas are displayed by smoothing operations and color intensity changes before the final map is rendered.

[0090] Figure 5 The model classification heat map of the scalp image data processing method provided in the first embodiment of the present invention is as follows: Figure 5 As shown in the figure, the image is a heat map visualization of the output of the scalp classification model. The text below the image is the true label of each image and the possible category predicted by the scalp classification model and the corresponding accuracy. The heat map effectively identifies key features such as black dot sign, yellow dot sign, and broken hair in alopecia areata, as well as key features such as diffusely distributed dot-like blood vessels in psoriasis, ignoring irrelevant details such as healthy hair.

[0091] Embodiment 2

[0092] Embodiment 2 of the present invention provides a processing device based on scalp image data. Figure 6 This is a schematic diagram of a scalp image data processing device according to a second embodiment of the present invention. Figure 6 As shown, the scalp image data-based processing device 400 includes a first data construction module 401 , a classification model construction module 402 , a scalp classification output module 403 , a second data construction module 404 , and an evaluation model construction module 405 .

[0093] The first data construction module 401 is used to obtain first sample data of scalp image data, where the first sample data includes multiple scalp types.

[0094] Among them, the various scalp types include: alopecia areata, androgenic alopecia, cicatricial alopecia, trichotillomania, psoriasis, seborrheic dermatitis, pigmented nevus, seborrheic keratosis, viral warts and non-scalp healthy skin.

[0095] The classification model building module 402 is used to build a pre-trained scalp classification model based on the SpectFormer model; train, verify and test the pre-trained scalp classification model through the first sample data to generate a scalp classification model.

[0096] Specifically, the pre-trained scalp classification model is trained through the training set in the first sample data to generate a scalp classification baseline model, wherein the resolution of the scalp image data in the training set is 224×224; the weights and bias of the baseline are interpolated to the scalp classification baseline model adapted to the resolution of 512×512 through the Bi cub ic interpolation method; the parameters of the baseline are frozen, and the parameters related to the bias, class embedding, normalization and scale factor are unfrozen; the scalp classification baseline model is trained, verified and tested by the first sample data converted into a resolution of 512×512 to generate a scalp classification model.

[0097] The scalp classification output module 403 is used to input the target type image data in the first sample data into the scalp classification model and perform scalp classification output, wherein the scalp classification output includes the real label of the image and the possible category predicted by the model and the corresponding accuracy rate.

[0098] The second data construction module 404 is used to generate second sample data according to the result output by the scalp classification.

[0099] Specifically, the entropy of the accuracy of each target type image data being alopecia areata, androgenic alopecia, and cicatricial alopecia is calculated after the scalp classification model is used; and multiple target type image data are arranged from large to small according to the calculated entropy.

[0100] The evaluation model construction module 405 is used to construct a pre-trained target type data evaluation model based on the YOLOv8 model; the pre-trained target type data evaluation model is trained, verified and tested through the image data with target type evaluation signs marked in the second sample data to generate a target type data evaluation model; wherein the second sample data is constructed using an active learning strategy.

[0101] Among them, the target type is alopecia areata, and the target type assessment signs include black dot sign, yellow dot sign, exclamation mark hair, broken hair, short vellus hair, and branching blood vessels.

[0102] The scalp classification output module 403 is further used to input the scalp image to be detected into the scalp classification model and output the scalp classification.

[0103] In an alternative approach, Figure 7 The second structural diagram of the processing device based on scalp image data provided in the second embodiment of the present invention is as follows: Figure 7 As shown, the scalp image data-based processing device 400 further includes a report automatic generation module 406 .

[0104] The automatic report generation module 406 is used to input the scalp image to be detected into the target type data evaluation model to generate target type evaluation signs if the scalp is classified as alopecia areata; determine the scalp active or inactive period according to the alopecia areata evaluation signs, and generate an evaluation report.

[0105] Among them, the active or inactive period of the scalp is determined according to the assessment signs of alopecia areata, specifically: if there is one or more of the assessment signs of alopecia areata including black spot sign, exclamation mark hair, broken hair and short vellus hair, the scalp is in the active period.

[0106] In an alternative approach, Figure 8 The third structural diagram of the processing device based on scalp image data provided in the second embodiment of the present invention is as follows: Figure 8 As shown, the scalp image data-based processing device 400 further includes a result visualization module 407 .

[0107] The result visualization module 407 is used to visualize the output of the scalp classification using the gradient weighted class activation map.

[0108] The scalp image data processing device provided in the second embodiment of the present invention corresponds to the scalp image data processing method provided in the first embodiment. The specific working process of each module has been described in detail in the method embodiment and will not be repeated here.

[0109] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0110] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0111] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A processing method based on scalp image data, characterized in that: The method comprises: Acquire first sample data of scalp image data, wherein the first sample data includes a plurality of scalp types; Building a pre-trained scalp classification model based on the SpectFormer model; training, verifying and testing the pre-trained scalp classification model using the first sample data to generate a scalp classification model; Inputting the target type image data in the first sample data into the scalp classification model, and performing scalp classification output, wherein the scalp classification output includes the true label of the image and the possible category predicted by the model and the corresponding accuracy rate; Generate second sample data according to the result output by the scalp classification; A pre-trained target type data evaluation model is constructed based on the YOLOv8 model; the pre-trained target type data evaluation model is trained, verified and tested through image data with target type evaluation signs marked in the second sample data to generate a target type data evaluation model; wherein the second sample data is constructed using an active learning strategy.

2. The processing method based on scalp image data according to claim 1, characterized in that: After the pre-trained target type data evaluation model is trained, verified and tested by the image data with target type evaluation signs marked in the second sample data to generate the target type data evaluation model, the method further includes: Inputting the scalp image data to be detected into the scalp classification model, and outputting the scalp classification; If the scalp is classified as a target type, the scalp image data to be detected is input into the target type data evaluation model, and a target type evaluation sign is output; according to the target type evaluation sign, it is determined whether the target type is in an active or inactive period, and an evaluation report is generated.

3. The processing method based on scalp image data according to claim 2, characterized in that: After inputting the scalp image data to be detected into the scalp classification model and outputting the scalp classification, the method further includes: The output of the scalp classification is visually interpreted using gradient-weighted class activation maps.

4. The processing method based on scalp image data according to claim 1, characterized in that: The target type is alopecia areata, and the target type assessment signs include black dot sign, yellow dot sign, exclamation mark hair, broken hair, short vellus hair, and branched blood vessels.

5. The processing method based on scalp image data according to claim 2, characterized in that: Determining whether the scalp is in an active or inactive phase according to the target type evaluation signs is specifically: If one or more of the target type assessment signs of black spot sign, exclamation mark hair, broken hair and short vellus hair are present, the scalp is in the active stage.

6. The processing method based on scalp image data according to claim 1, characterized in that: The various scalp types also include androgenic alopecia, cicatricial alopecia, trichotillomania, psoriasis, seborrheic dermatitis, pigmented nevus, seborrheic keratosis, viral warts and non-scalp healthy skin.

7. The processing method based on scalp image data according to claims 4 and 6, characterized in that: The generating second sample data according to the result of the scalp classification output specifically includes: Calculating the entropy of the accuracy rate of each target type image data after passing through the scalp classification model, the possible types are alopecia areata, androgenic alopecia and cicatricial alopecia; Arrange multiple target type image data from large to small according to the calculated entropy.

8. The processing method based on scalp image data according to claim 1, characterized in that: The step of training, verifying and testing the pre-trained scalp classification model using the first sample data to generate the scalp classification model specifically includes: Training the pre-trained scalp classification model using the training set in the first sample data to generate a scalp classification baseline model, wherein the resolution of the scalp image data in the training set is 224×224; The baseline weights and biases are interpolated to the scalp classification baseline model adapted to the 512×512 resolution through the Bicubic interpolation method; Freeze the parameters of the baseline and unfreeze the parameters related to bias, class embedding, normalization, and scaling factors; The scalp classification baseline model is trained, verified and tested by converting the first sample data into a 512×512 resolution to generate a scalp classification model.

9. A processing device based on scalp image data, characterized in that: The device comprises: A first data construction module, used to obtain first sample data of scalp image data, wherein the first sample data includes multiple scalp types; A classification model building module, used to build a pre-trained scalp classification model based on the SpectFormer model; train, verify and test the pre-trained scalp classification model through the first sample data to generate a scalp classification model; A scalp classification output module, used to input the target type image data in the first sample data into the scalp classification model and perform scalp classification output, wherein the scalp classification output includes the real label of the image and the possible category predicted by the model and the corresponding accuracy rate; A second data construction module, used to generate second sample data according to the result output by the scalp classification; An evaluation model construction module is used to construct a pre-trained target type data evaluation model based on the YOLOv8 model; the pre-trained target type data evaluation model is trained, verified and tested through image data with target type evaluation signs marked in the second sample data to generate a target type data evaluation model; wherein the second sample data is constructed using an active learning strategy.

10. The processing method based on scalp image data according to claim 9, characterized in that: The device also includes: A result visualization module is used to visualize the output of the scalp classification using gradient weighted class activation mapping.

Images