Face demodex area detection method, device and system and storage medium

Through the deep neural network model, especially the Transformer bidirectional jump connection model, combined with image preprocessing and data enhancement technology, the problems of complex patterns and edge recognition in face demodex detection are solved, significantly improving detection accuracy and efficiency, and realizing the Automated and efficient detection of demodex mite in human face.

CN119068524APending Publication Date: 2024-12-03赵军
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411039165.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art has problems such as image quality influence, environmental factor interference, individual differences and high equipment costs in the detection of human face demodex, resulting in low detection accuracy and efficiency, and the inability to effectively deal with morphological characteristics and edge recognition of complex patterns.

Method used

Using deep neural network model, especially Transformer bidirectional jump connection model, combined with image preprocessing and data enhancement technology, a significant demodex detection module is built, and the demodex features are automatically learned through multi-layer convolution and pooling operations, and the complex processing is processed. mode and improve detection accuracy.

Benefits of technology

It significantly improves the accuracy and efficiency of the detection of the demodex mite area in human face, reduces false alarms and missed reports, enhances the robustness of the model and the ability to identify complex patterns, realizes automated detection, reduces human intervention, and improves The objectivity and convenience of diagnosis are achieved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068524A_ABST
    Figure CN119068524A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of image processing, in particular to a face demodex area detection method, device and system and a storage medium. The method comprises the following steps: acquiring a plurality of face demodicid mite images by using an image acquisition module; marking the areas of the face demodex mites, and constructing a data set; constructing a deep neural network model, and capturing a complex mode of the face demodicid mites in the image data; training a deep neural network model through the marked color image of the face demodicus, comparing the difference between a predicted value and a true value image through a constructed loss function, and carrying out iterative calculation through a back propagation algorithm to adjust the weight of the model; a deep neural network model is adopted to process the face image which is not manually marked, and a face demodicid mite area detection result of the to-be-detected image is obtained; outputting a face demodex area detection result of the to-be-detected image through an output module; the method has the advantages that human intervention is reduced, the diagnosis objectivity is improved, and the detection efficiency and accuracy are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, system and storage medium for detecting Demodex mite areas on a human face. Background Art

[0002] Demodex on the face plays an important role in the study of human skin diseases. These mites can cause skin inflammation, itching and other problems, which seriously affect people's daily quality of life. However, most of the current detection methods for Demodex on the face rely on traditional medical collection and laboratory testing methods, which are not only time-consuming and labor-intensive, but also inconvenient for patients. Therefore, developing an efficient and convenient method for detecting Demodex on the face has become an important research topic in the field of skin disease diagnosis. With the continuous advancement of science and technology, artificial intelligence and image processing technology have provided new possibilities for achieving this goal. By using deep learning algorithms and combining a large number of image data sets, a model that can automatically identify and accurately locate Demodex on the face can be trained. This method not only significantly improves the accuracy and efficiency of detection, but also reduces the interference of human factors, making the detection results more objective and reliable.

[0003] Studies have shown that due to the special properties of human face Demodex mites, the captured image data of human face Demodex mites contain a variety of complex patterns, mainly including but not limited to the following five:

[0004] 1. Morphological characteristics diversity pattern: The morphological characteristics diversity pattern specifically refers to the shape, size, position and orientation of Demodex mites which may vary from individual to individual.

[0005] 2. Background and target distinction mode: Background and target distinction mode specifically refers to skin texture, hair, skin lesions (such as erythema, papules) and other biological factors that may interfere with the identification of Demodex mites. The lighting conditions, color saturation, contrast and resolution in the image may also affect the visibility and identification of Demodex mites.

[0006] 3. Density and aggregation pattern: The density and aggregation pattern specifically means that Demodex mites may be densely distributed in some areas and sparsely distributed in other areas, forming an aggregation effect. This uneven distribution may be related to the physiological state of the skin (such as oil secretion and sweat gland activity).

[0007] 4. Dynamic change pattern: The dynamic change pattern specifically means that the location, number and behavior of Demodex mites may change over time, especially before and after treatment or when environmental conditions change.

[0008] 5. Co-existing pathological condition pattern: The co-existing pathological condition pattern specifically states that Demodex may coexist with other skin conditions, such as acne, rosacea, or seborrheic dermatitis, which requires the model to distinguish between the symptoms caused by Demodex and the symptoms of other skin diseases.

[0009] In the current technical literature, there has been no special research on the complex patterns displayed by Demodex mites on the human face. The neural network models used in similar disease research often adopt a one-way feature propagation mechanism from the bottom layer to the top layer. Although this method solves the problem of single feature recognition of Demodex-like diseases to a certain extent, it is prone to information loss, insufficient information sharing, and reduced image resolution during the feature transmission process.

[0010] More specifically, for the morphological features of the complex patterns of Demodex mites that this patent focuses on, that is, the feature diversity pattern recognition problem, when processing images containing irregular edges of Demodex mites, the existing neural network Demodex detection model lacks effective sequence encoding of facial regions, making it difficult for the model to effectively process these complex Demodex image samples. This in turn results in the model being unable to obtain the rich and rare salience features and edge contour features of the mite location, ultimately affecting the overall accuracy of the model. Therefore, accurately capturing and analyzing complex patterns in Demodex mites images on the human face remains a very challenging task for neural network models.

[0011] For example, a Chinese patent with application number 2021114895341 and application date of December 8, 2021, is an invention patent named "Non-contact testing method, system, storage medium and processor for skin mites". Its technical solution is: obtain a skin image to be measured; input the skin image into a mite image recognition model, and extract identification features of the skin image through the mite image recognition model to determine the mite identification features contained in the skin image; use a single identification feature segmentation method to determine the number of target identification area image units containing a single mite identification feature; segment the skin image to be measured according to the single mite identification feature principle to ensure that the segmented image unit only contains a single mite identification feature, and determine the number of mites contained in the skin image based on the number of segmented target identification area image units.

[0012] Although the above patent can realize non-contact detection of Demodex mites on the skin, the single mite identification feature principle of the above patent method has the following problems, including:

[0013] First, the image quality has an impact. The imaging quality of non-contact image acquisition equipment directly affects the accuracy of recognition. When the image resolution is low, the single mite recognition feature may not be clearly presented, making it difficult for the model to accurately identify the shape and number of mites. Noise in the image (such as uneven lighting, dust, natural skin texture, etc.) may be mistaken for a single mite recognition feature, thereby affecting the accuracy of recognition. Blurred or inaccurate image focus will make the outline of the mite unclear, affecting the effect of extracting a single mite recognition feature.

[0014] Secondly, there is the influence of environmental factors. The light, background and other factors in the image acquisition environment may interfere with image acquisition and recognition, affecting the final detection results of a single mite recognition feature. Strong light or shadows may cause overexposure or underexposure of the mite feature, affecting recognition. Complex backgrounds or colors similar to mites may interfere with the model's recognition of a single mite recognition feature, resulting in false positives or negative positives. In dynamic environments, such as wind, vibration or equipment movement, images may become unstable, affecting image quality.

[0015] The second is individual differences. Different skin conditions (such as skin color, skin texture, etc.) of different individuals may affect the recognition of a single mite identification feature, resulting in deviations in the test results. Different skin colors may affect the visibility of mite features, especially the recognition of light-colored mites on dark skin. Even natural textures or lesions on the skin (such as acne and scars) may be mistakenly identified as mites.

[0016] The accuracy and resolution of the image acquisition equipment (such as optical lens modules) used in the end will directly affect the accuracy of single mite identification features; the equipment is expensive, and high-precision optical lens modules and sensors may be expensive, limiting the popularity of the equipment; professional equipment also requires regular maintenance and calibration to ensure long-term accuracy and stability.

[0017] In summary, the above patent adopts a method based on the principle of a single mite identification feature, but the model structure of this method is single and cannot capture spatial features. In addition, it is difficult to collect data sets and there is a lack of data enhancement methods. As a result, the above patent cannot be specifically adapted to capture the complex patterns of facial Demodex mites in image data. Summary of the invention

[0018] In order to solve the above problems existing in the prior art, the present application provides a method, device, system and storage medium for detecting facial Demodex areas with complex pattern characteristics of facial Demodex images with higher detection efficiency, accuracy and convenience.

[0019] The technical solutions provided by this application are as follows:

[0020] In a first aspect, a method for detecting Demodex mites in a human face is provided, comprising the following specific steps:

[0021] Step 1: Use the image acquisition module to obtain multiple images of Demodex mites on the human face;

[0022] Step 2: Annotate the Demodex regions in all facial Demodex images to construct a facial Demodex image dataset;

[0023] Step 3: Build a deep neural network model to capture the complex patterns of facial Demodex in image data;

[0024] Step 4: The labeled color image of human face demodex mites is used to train the deep neural network model, and the difference between the predicted value and the true value image is compared through the constructed loss function. The back propagation algorithm is used to iteratively calculate and adjust the model's own weights to improve the accuracy of predicting the human face demodex mites area;

[0025] Step 5: Use the trained deep neural network model to process the face image that has not been manually labeled to obtain the face demodex area detection result of the image to be tested;

[0026] Step 6: Output the result of the facial Demodex region detection of the image to be tested through the output module.

[0027] Furthermore, complex patterns include patterns of morphological feature diversity, patterns of background and target distinction, patterns of density and aggregation, patterns of dynamic changes, and patterns of coexisting pathological conditions.

[0028] Furthermore, in step one, multiple facial Demodex images are processed by a preprocessing module; after the annotation is completed in step two, multiple truth maps are generated; in step three, the deep neural network model is a deep neural network model with a Transformer bidirectional skip connection, and the saliency Demodex detection module composed of this deep neural network model captures the complex patterns of facial Demodex in the image data in step three; in step five, the trained Transformer bidirectional skip connection deep neural network model is used to process the facial color images that have not been manually labeled.

[0029] Furthermore, the specific steps of constructing a Transformer bidirectional skip connection deep neural network model in step 3 are:

[0030] Step a: using multiple labeled facial Demodex images, and converting the feature maps obtained by the input module into a feature matrix;

[0031] Step b: The encoder module extracts high-order mite region features layer by layer, the feature map size gradually decreases, and the spatial dimension gradually increases, obtaining a high-dimensional low-rank compressed feature matrix;

[0032] Step c: The output feature map of the encoder module is passed through the sequence prediction module to predict the spatial sequence position feature matrix of Demodex;

[0033] Step d: The spatial sequence position feature matrix is ​​restored to the feature map size layer by layer through the decoder module, and then passed through the content module to obtain a decoded feature map of the same size as the feature map of the input module;

[0034] Step e: extracting the features of the decoder module process layer by layer in the decoder module, and performing the inversion connection operation with the feature map of the same layer of the encoder module in the inversion effective mite area feature connection module, and obtaining the extracted feature maps of different scales through the layered extraction feature module;

[0035] Step f: After the extracted feature maps of different scales are stacked with the decoded feature maps output by the content module, they are fused into feature maps of uniform size through a feature fusion module to obtain a predicted feature map;

[0036] Step g: The predicted feature map is outputted by the feature fusion module to obtain the significant area of ​​the mite area, and the deep neural network model is used to form a significant demodex detection module to capture the complex patterns in the facial demodex image data.

[0037] Furthermore, in step a, the specific formula for obtaining the feature map through the input module is as follows:

[0038] F i =Max(Conv(I input ))

[0039] In the formula, I input is the input image, F i is the feature map output by the i-th layer, Max is the maximum pooling layer function, and Conv is the convolution function of the convolution layer;

[0040] In step b, the specific formula for extracting high-order mite area features layer by layer by the encoder module is as follows:

[0041]

[0042] In the formula, b is the extracted b-th layer feature, Encoder b () is the encoder function of the b-th layer, En b-1 is the output feature of the previous layer of level b, En b is the output feature of the encoding module at level b, and e is the number of extracted feature layers;

[0043] In step c, the output feature graph of the encoder module is passed through the sequence prediction module to obtain the specific formula for predicting the spatial sequence position feature matrix of Demodex mites as follows:

[0044] Trans=Transformer(En b )

[0045] Where Trans is the spatial sequence position feature matrix, Transformer() is the Transformer model function;

[0046] In the step e, the high-order features of Demodex and the spatial sequence position feature matrix are decoded layer by layer in the decoder module, and the high-dimensional low-rank features are gradually restored to the low-dimensional Demodex significant area. The specific formula is as follows:

[0047]

[0048] In the formula, Decoder b () is the decoder module function of the b-th layer, De b Output features for the decoding module at level b;

[0049] In step e, the specific calculation formula for performing the inversion operation in the inversion effective mite area feature connection module is as follows:

[0050]

[0051] In the formula, Extract() is the function of the extraction module, which consists of 1 convolution layer, S b This is the feature map after inverting the effective Demodex area. is the element-by-element multiplication of the feature matrix, Add element-wise to the feature matrix;

[0052] In step f, the specific formula for the decoded feature map of the same size as the feature map of the input module obtained through the content module is as follows:

[0053] F c =Content(Decoder(Transformer(Encoder(F i ))))

[0054] Among them, Content() represents the content module function, Decoder() represents the decoder module function, Transformer() represents the Transformer module function, Encoder() represents the encoder module function, and F c is the output feature of the content module;

[0055] In step f, the extracted feature maps of different scales are fused into a feature map of uniform size through a feature fusion module, and are stacked with the decoded feature map output by the content module. The mean of the dimensions is taken, and the specific calculation formula for the predicted feature map is obtained by using the mean strategy as follows:

[0056]

[0057] In the formula, the feature map is obtained by feature fusion after the encoder module and the decoder module are inverted. F c is the output feature of the content module, Concate() is the cascade stacking function, Mean() is to calculate the average value of elements along the channel dimension, F fuse The obtained feature fusion map.

[0058] Furthermore, in step 4, the saliency map loss function is constructed to train the neural network using the Adam optimizer; the back propagation algorithm is used for iterative calculation, firstly the gradient of the back propagation of the neural network is calculated, and the total model loss composed of the content loss and the extraction loss is introduced, corresponding to the forward connection subnet and the reverse connection subnet respectively; the model parameters θ include weights W1, ..., W L and biases b1,…,b L ; The binary cross entropy loss σ(θ) is used to evaluate the difference between the predicted saliency map and the true value map, and the gradient adjustment parameters of the back propagation calculation model are expressed as follows:

[0059] σ(GT,·|θ)=Ylog H L +(1-Y)log(1-H L ), Y∈{0,1}

[0060] Where σ(GT,·|θ) is the distance between the true image GT and the predicted image, and H L The probability that sample L is predicted as a positive class, Y is the sign function, which takes 1 when sample L is a positive class and 0 otherwise, and θ is the model parameter to be adjusted.

[0061] In a second aspect, a device for detecting Demodex mites in a human face is provided, comprising:

[0062] An image acquisition module, used for acquiring color images of Demodex mites on the human face;

[0063] A preprocessing module is used to process the color image of Demodex mites on the human face by resizing the image, normalizing the pixel values, and cropping and flipping the image;

[0064] The significant demodex detection module is used to capture the complex patterns in the image data to be tested and calculate the regional detection results of demodex mites on the human face in the image to be tested;

[0065] The output module is used to output the extracted Demodex area on the human face.

[0066] Furthermore, the significant Demodex detection module includes:

[0067] The input module, including multiple convolutional layers and maximum pooling, is used to preprocess the input original human face demodex mite image; in this process, the resolution of input and output will be consistent with the original image to ensure the quality and clarity of the image;

[0068] The encoder module includes multiple encoders and is composed of multiple convolutional layers, batch normalization, and ReLU activation layers. It is used to extract the characteristic functions of Demodex mites on the face and gradually obtain high-order semantic features. The multi-layer convolutional layers can extract various features and patterns of Demodex mites from the image.

[0069] The decoder module includes multiple decoders and is composed of multiple layers of deconvolution, which is used for low-order demodex information reconstruction and pixel-level binary classification. The high-level semantic features extracted by the encoding module are gradually restored back to the original face infection image, and the image area is classified at the same time. The deconvolution layer maps the high-level features back to the original image space to restore the details and structure of the image.

[0070] The sequence prediction module, which consists of Transformer modules, is used to perform facial region sequence encoding on the compressed feature matrix output by the encoder model, and treats each row of Demodex features as a spatial sequence. Through sequence analysis of the feature matrix, the sequence prediction module captures the spatial variation and trend of Demodex in different facial regions, thereby accurately predicting the number and distribution of Demodex.

[0071] Content module, a module for post-processing the output predicted image, with the input and output resolutions being consistent with the original image; this module performs post-processing on the output predicted human face demodex mite image, including color correction, denoising and sharpening;

[0072] A reverse effective mite region feature connection module is used to perform a reverse connection operation between the encoder module and the decoder module;

[0073] A hierarchical feature extraction module is used to extract the significant Demodex features from the encoder module and the decoder module at the same level;

[0074] The feature fusion module adopts the mean strategy to merge the output Demodex feature maps at different levels to obtain the output results.

[0075] In a third aspect, a system for detecting facial Demodex mites areas is provided, comprising a server and a client, wherein the server comprises a processor and a memory; the memory stores facial Demodex mites image data, program codes and computer executable instructions, which when executed by the processor are used to implement any facial Demodex mites area detection method of the first aspect; the client is used to upload images and display results.

[0076] In a fourth aspect, a storage medium for detecting facial Demodex areas is provided, which is used to store facial Demodex image data, program codes, and computer executable instructions on a computer, and to process facial Demodex image data by running the program code stored on the computer to implement the facial Demodex area detection method described in any one of the first aspects.

[0077] According to the above technical solution, the beneficial effects of this application are as follows:

[0078] 1. The method for detecting the Demodex area on the human face of the present invention not only relies on a deep neural network model, but also further combines image processing technology and a medical knowledge base to ensure the accuracy and efficiency of the complex patterns of the detection results. The deep neural network model is introduced to optimize the characteristics of Demodex on the human face, including morphological feature diversity patterns, background and target distinction patterns, density and aggregation patterns, dynamic change patterns, and coexistence of pathological conditions patterns; through multi-level convolution operations and pooling operations, the deep neural network model can automatically learn the subtle texture and shape characteristics of Demodex on the human face skin, thereby achieving accurate positioning of the Demodex area.

[0079] 2. The method for detecting the Demodex region on the human face of the present invention increases the robustness of the model for identifying complex patterns of Demodex mites. By introducing a variety of preprocessing methods for Demodex mites on the human face, including but not limited to random cropping of the image center, random channel discarding, sharpening, pixel loss, flipping, grid distortion, inverted image and superpixel fusion, etc., color face images are processed to generate cases with unclear boundaries and various shapes, thereby ensuring that the model can more accurately identify and reduce false alarms (false positives) and omissions (false negatives) when facing complex patterns of Demodex mites in the real world.

[0080] 3. The method for detecting facial Demodex mites of the present invention adopts the U-Net architecture as the basis of the basic network structure, combines the reverse region connection as a means of feature reuse, and also adopts a more complex skip connection mechanism, which includes both skip connections within the module and connections between modules; therefore, multiple features such as shape, texture, color extracted by multi-layer feature maps are subjected to reverse feature map connection and feature fusion, so that the model can dynamically adjust the degree of attention to different features according to actual conditions, rather than just relying on boundary information; in addition, the complex skip connection method helps to compensate for the information loss that may be generated by the neural network during the maximum pooling and convolution operations, thereby ensuring the prediction accuracy of the complex pattern of facial Demodex mites of the model.

[0081] 4. The method for detecting the Demodex region on the human face of the present invention is aimed at expressing the spatial sequence position of the complex pattern of Demodex in the human face image; for the sequence encoding of the human face region, the Transformer model structure is introduced, which improves the performance index in the detection of the Demodex region, especially when processing the multi-point and diffuse spatial distribution characteristics of the Demodex on the human face, it can more accurately capture and analyze the feature details, thereby achieving a better detection effect.

[0082] 5. The method and device for detecting the Demodex area on the human face of the present invention can automatically detect and extract the Demodex area on the human face, reduce human intervention, improve the objectivity of diagnosis, greatly improve the detection efficiency and accuracy, and have significant advantages over the medical collection and testing methods in the prior art.

[0083] 6. The facial demodex area detection system of the present invention can display the detection results in real time, so that users can understand their health status in time. Users only need to upload facial images through the client, and the system can automatically detect the facial demodex area and display the results on the client. This provides users with a convenient health management method and helps to detect and intervene in diseases at an early stage.

[0084] 7. The present invention also provides a detection storage medium for the demodex area of ​​the human face, so that the image data of demodex mites on the human face, program codes and computer executable instructions can be conveniently stored on a computer, making it convenient to review and analyze the detection results at different locations or at different times, thereby improving the convenience and flexibility of medical services. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 Schematic diagram of data enhancement of the method of the present invention.

[0086] Figure 2 It is a schematic diagram of the Demodex area detection model on the human face.

[0087] Figure 3 This is a schematic diagram of the internal repetition and connection between the encoder and decoder.

[0088] Figure 4 It is a schematic diagram of the inverted effective mite region feature connection module.

[0089] Figure 5 This is a comparison chart of the expert annotations of the face mite dataset and the detection area of ​​this method.

[0090] Figure 6 It is a practical application flow chart of the method of the present invention.

[0091] Figure 7 It is an application relationship diagram of the system of the present invention. DETAILED DESCRIPTION

[0092] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0093] Example 1

[0094] A method for detecting the Demodex area on a human face comprises the following specific steps:

[0095] Step 1: Use the image acquisition module to obtain multiple images of Demodex mites on the human face. The multiple images of Demodex mites on the human face collected include skin images of various types of mites, skin images with different environmental variables, and skin images with skin color and texture changes. Skin images of various types of mites include, for example, Demodex mites, dust mites, etc., to ensure that the model can identify different types of mites. Skin images with different environmental variables are skin images collected under different lighting conditions and background environments, covering multiple scenes such as daytime, nighttime, indoors, and outdoors, so as to improve the environmental adaptability of the model; skin images with texture changes are samples of different skin colors and skin textures (such as dryness, oiliness, sensitivity, etc.) to cover a wide range of individual differences;

[0096] Step 2: Finely label the mite areas in all facial Demodex images to ensure that the model can accurately identify the mite boundaries; Figure 1 As shown in the figure, in order to construct a diverse dataset of facial Demodex mites images, a variety of data enhancement strategies were used, including super contrast equalization, center cropping, channel discarding, sharpening, pixel discarding, flipping, grid distortion, inverted image and superpixel techniques to increase the diversity of the dataset and reduce the risk of overfitting;

[0097] Step 3: Build a deep neural network model, which uses convolutional neural network (CNN), U-Net, etc. to perform feature extraction and image segmentation to capture the complex patterns of facial Demodex in image data;

[0098] Step 4: The labeled color image of facial Demodex is used to train the deep neural network model. The difference between the predicted value and the true value image is compared through the constructed loss function. The back propagation algorithm is used to iteratively calculate and adjust the weight of the model itself to improve the accuracy of predicting the facial Demodex area. The deep neural network model training adopts loss function design and regularization technology. The loss function involves selecting a suitable loss function, such as cross entropy, Dice Loss, IoU Loss, etc., to promote model convergence; regularization technology includes applying Dropout, L1 / L2 regularization, etc. to prevent model overfitting;

[0099] Step 5: Use the trained deep neural network model to process the face image that has not been manually labeled to obtain the face demodex area detection result of the image to be tested;

[0100] Step 6: Output the result of the facial Demodex region detection of the image to be tested through the output module.

[0101] The deep neural network model is evaluated by evaluating the model performance on an independent test set to ensure its generalization ability, and then the model parameters are continuously adjusted based on feedback from actual applications to improve the detection accuracy.

[0102] Example 2

[0103] Based on Example 1, in step 1, multiple facial Demodex images are processed by a preprocessing module; after the annotation is completed in step 2, multiple truth maps are generated; in step 3, the deep neural network model is a deep neural network model with a Transformer bidirectional skip connection, and the complex pattern of facial Demodex in the image data in step 3 is captured by the saliency Demodex detection module composed of this deep neural network model; in step 5, the trained Transformer bidirectional skip connection deep neural network model is used to process the facial color image that has not been manually labeled.

[0104] The complex patterns include morphological diversity patterns, background and target distinction patterns, density and aggregation patterns, dynamic change patterns and coexisting pathological conditions patterns; the morphological diversity patterns specifically refer to the shape, size, position and direction of Demodex mites, which may vary due to individual differences; the background and target distinction patterns specifically refer to skin texture, hair, skin lesions (such as erythema, papules) and other biological factors that may interfere with the identification of Demodex mites, and the lighting conditions, color saturation, contrast and resolution in the image may also affect the visibility and identification of Demodex mites; the density and aggregation patterns specifically refer to the fact that Demodex mites may be densely distributed in some areas and sparsely distributed in other areas, forming an aggregation effect, and this uneven distribution may be related to the physiological state of the skin (such as oil secretion, sweat gland activity); the dynamic change patterns specifically refer to the fact that the position, number and behavior of Demodex mites may change over time, especially before and after treatment or when environmental conditions change; the coexisting pathological conditions patterns specifically refer to the fact that Demodex mites may coexist with other skin conditions, such as acne, rosacea or seborrheic dermatitis, which requires the model to distinguish between the symptoms caused by Demodex mites and the symptoms of other skin diseases.

[0105] To accurately capture the complex patterns of Demodex mentioned above, e.g. Figure 2 As shown in the figure, this method uses the U-Net architecture on the basic network structure. The specific steps of building a complex model in step 3 - the Transformer bidirectional jump connection deep neural network model are:

[0106] Step a: using multiple labeled facial Demodex images, and converting the feature maps obtained by the input module into a feature matrix;

[0107] Step b: The encoder module extracts high-order mite region features layer by layer, the feature map size gradually decreases, and the spatial dimension gradually increases, obtaining a high-dimensional low-rank compressed feature matrix;

[0108] Step c: The output feature map of the encoder module is passed through the sequence prediction module to predict the spatial sequence position feature matrix of Demodex;

[0109] Step d: The spatial sequence position feature matrix is ​​restored to the feature map size layer by layer through the decoder module, and then passed through the content module to obtain a decoded feature map of the same size as the feature map of the input module;

[0110] Step e: extracting the features of the decoder module process layer by layer in the decoder module, and performing the inversion connection operation with the feature map of the same layer of the encoder module in the inversion effective mite area feature connection module, and obtaining the extracted feature maps of different scales through the layered extraction feature module;

[0111] Step f: After the extracted feature maps of different scales are stacked with the decoded feature maps output by the content module, they are fused into feature maps of uniform size through a feature fusion module to obtain a predicted feature map. The multi-scale features are combined to improve the model's ability to identify mites of different sizes.

[0112] Step g: The predicted feature map is outputted by the feature fusion module to obtain the significant area of ​​the mite area, and the deep neural network model is used to form a significant demodex detection module to capture the complex patterns in the facial demodex image data.

[0113] In step a, the specific formula for obtaining the feature map through the input module is as follows:

[0114] F i =Max(Conv(I input ))

[0115] In the formula, I input is the input image, F i is the feature map output by the i-th layer, Max is the maximum pooling layer function, and Conv is the convolution function of the convolution layer;

[0116] In step b, the specific formula for extracting high-order mite area features layer by layer by the encoder module is as follows:

[0117]

[0118] In the formula, b is the extracted b-th layer feature, Encoder b () is the encoder function of the b-th layer, En b-1is the output feature of the previous layer of level b, En b is the output feature of the encoding module at level b, and e is the number of extracted feature layers;

[0119] In step c, the output feature map of the encoder module is passed through the sequence prediction module to obtain the specific formula for predicting the spatial sequence position feature matrix of Demodex mites as follows:

[0120] Trans=Transformer(En b )

[0121] Where Trans is the spatial sequence position feature matrix, Transformer() is the Transformer model function;

[0122] In step e, the high-order features of Demodex mites and the spatial sequence position feature matrix are decoded layer by layer in the decoder module, and the high-dimensional low-rank features are gradually restored to the low-dimensional mite salient area. The specific formula is as follows:

[0123]

[0124] In the formula, Decoder b () is the decoder module function of the b-th layer, De b Output features for the decoding module at level b;

[0125] In step e, the specific calculation formula for performing the inversion operation in the inversion effective mite area feature connection module is as follows:

[0126]

[0127] In the formula, Extract() is the function of the extraction module, which consists of 1 convolution layer, S b This is the feature map after inverting the effective Demodex area. is the element-by-element multiplication of the feature matrix, Add element-wise to the feature matrix;

[0128] In step f, the specific formula for the decoded feature map of the same size as the feature map of the input module obtained through the content module is as follows:

[0129] F c =Content(Decoder(Transformer(Encoder(F i ))))

[0130] Among them, Content() represents the content module function, Decoder() represents the decoder module function, Transformer() represents the Transformer module function, Encoder() represents the encoder module function, and F c is the output feature of the content module;

[0131] In step f, the extracted feature maps of different scales are fused into a feature map of uniform size through a feature fusion module, and are stacked with the decoded feature map output by the content module. The mean of the dimensions is taken, and the specific calculation formula for the predicted feature map is obtained by using the mean strategy as follows:

[0132]

[0133] In the formula, the feature map is obtained by feature fusion after the encoder module and the decoder module are inverted. F c is the output feature of the content module, Concate() is the cascade stacking function, Mean() is to calculate the average value of elements along the channel dimension, F fuse The obtained feature fusion map.

[0134] In the step 4, the saliency map loss function is constructed and the Adam optimizer is used to train the neural network; the back propagation algorithm is used for iterative calculation, and the gradient of the back propagation of the neural network is first calculated, and the total model loss composed of the content loss and the extraction loss is introduced, which correspond to the forward connection subnet and the reverse connection subnet respectively; the model parameters θ include weights W1, ..., W L and biases b1,…,b L ; The binary cross entropy loss σ(θ) is used to evaluate the difference between the predicted saliency map and the true value map, and the gradient adjustment parameters of the back propagation calculation model are expressed as follows:

[0135] σ(GT,·|θ)=Ylog H L +(1-Y)log(1-H L ), Y∈{0,1}

[0136] Where σ(GT,·|θ) is the distance between the true image GT and the predicted image, and H L The probability that sample L is predicted as a positive class, Y is the sign function, which takes 1 when sample L is a positive class and 0 otherwise, and θ is the model parameter to be adjusted.

[0137] Example 3

[0138] Based on Example 1 and Example 2, a detection device for the Demodex area on a human face includes:

[0139] An image acquisition module, used for acquiring color images of Demodex mites on the human face;

[0140] A preprocessing module is used to process the color image of Demodex mites on the human face by resizing the image, normalizing the pixel values, and cropping and flipping the image;

[0141] The significant demodex detection module is used to capture the complex patterns in the image data to be tested and calculate the regional detection results of demodex mites on the human face in the image to be tested;

[0142] The output module is used to output the extracted Demodex area on the face; the output results will be displayed on the client and terminal for reference by relevant personnel.

[0143] The significant Demodex detection module includes:

[0144] The input module includes multiple convolutional layers and maximum pooling, which are used to preprocess the input original human face demodex mite image. In this process, the resolution of the input and output will be consistent with the original image, ensuring the quality and clarity of the image, and can effectively remove noise and interference in the image, improving the clarity and quality of the image.

[0145] The encoder module includes multiple encoders and is composed of multiple convolutional layers, batch normalization and ReLU activation layers, which are used to extract the characteristic functions of Demodex mites on the face and gradually obtain high-order semantic features; multiple convolutional layers can extract various features and patterns of Demodex mites from images.

[0146] The decoder module includes multiple decoders and is composed of multiple layers of deconvolution, which is used for low-order demodex information reconstruction and pixel-level binary classification. The main function of this module is to gradually restore the high-level semantic features extracted by the encoding module back to the original face infection image and classify the image area; the deconvolution layer maps the high-level features back to the original image space to restore the details and structure of the image.

[0147] The sequence prediction module, which consists of Transformer modules, is used to perform facial region sequence encoding on the compressed feature matrix output by the encoder model, and treats each row of Demodex features as a spatial sequence. Through sequence analysis of the feature matrix, the sequence prediction module captures the spatial variation and trend of Demodex in different facial regions, thereby accurately predicting the number and distribution of Demodex.

[0148] Content module, a module for post-processing the output predicted image, with the input and output resolutions being consistent with the original image; this module performs post-processing on the output predicted facial Demodex mite image, including color correction, denoising and sharpening, to further improve the image quality and clarity, and makes the output image more realistic and vivid through post-processing technology;

[0149] A reverse effective mite region feature connection module is used to perform a reverse connection operation between the encoder module and the decoder module;

[0150] The hierarchical feature extraction module is used to extract the significant Demodex features of the encoder module and decoder module at the same level. These feature information can better reflect the important information and features in the image, and provide an important basis for the subsequent classification of infected pixels and identification of Demodex.

[0151] The feature fusion module adopts the mean strategy to merge the output Demodex feature maps of different levels to obtain the output results; this fusion method can effectively improve the accuracy of image classification and recognition, and provide more reliable support for subsequent applications.

[0152] Example 4

[0153] A system for detecting facial demodex areas comprises a server and a client, wherein the server comprises a processor and a memory; the memory stores facial demodex image data, program codes and computer executable instructions, which are used to implement any of the methods described in Embodiment 1 or 2 when executed by the processor; and the client is used to upload images and display results.

[0154] Example 5

[0155] A detection storage medium for facial demodex areas, used for storing facial demodex image data, program codes and computer executable instructions on a computer, and processing facial demodex image data by running the program codes stored on the computer, so as to implement any of the methods described in embodiment 1 or 2.

[0156] Example 6

[0157] On the basis of Examples 1 to 5, a specific example is provided, in which an image recognition model capable of identifying human Demodex is constructed and relevant data is collected; these data should include images of human Demodex under different conditions to ensure that the model can learn and identify the characteristics of Demodex; the data is divided into two parts, the first part of the data set is a total of 47 images obtained by actually visiting patients who use mite removal drugs and obtaining image authorization, the age distribution of the patients ranges from 18 to 77 years old, and the image size and resolution vary; the second part of the data set is a total of 500 images collected from the Internet about common named diseases of human Demodex, such as Demodex, rosacea, acne, and acne.

[0158] Data preparation: Cleaning, formatting, organizing, and labeling the collected image data; this includes adjusting image size, contrast, and brightness; standardizing the data, removing blurry or corrupted images, and dividing the data into training, validation, and test sets;

[0159] Convert the image type and perform data enhancement. Convert the png format images in the data set to jpg format. Use image data enhancement to increase the size of the data set.

[0160] To improve the accuracy of the model, we combined the annotations of multiple experts to build consensus and identify possible difficulties. We used the open source image annotation tool LabelMe software and invited 5 industry experts to annotate the same image at the same time. The average overlapping range of the 5 annotations was used as the significant area of ​​the face mite area and saved as a json file.

[0161] Use the open source third-party library labelme-toolkit to convert the labeled json file into a salient area image.

[0162] After the data is prepared, a deep neural network model for identifying Demodex images is constructed, and the training data is used for training. A bidirectional skip connection deep neural network model is constructed. The training model is used to accurately detect and extract the Demodex area on the face, as follows:

[0163] The purpose of the input module is to preprocess the input image to obtain the initial features, which is conducive to unifying the distribution of image features and providing input information of the same size for subsequent modules; the initial input module consists of a convolutional layer and a maximum pooling layer, and the input json is converted into an image I input Get the initial feature F i , which is conducive to unifying the distribution of facial demodex features and providing the same size of input information for subsequent modules. The initial input module consists of multiple convolutional layers and maximum pooling, the configuration is shown in Table 1, and the calculation formula is expressed as:

[0164] F i =Max(Conv(I input ))

[0165] In the formula, I input is the input image, F i is the feature map output by the i-th layer, Max is the maximum pooling layer function, and Conv is the convolution function of the convolution layer.

[0166] Table 1 Configuration parameters of input module

[0167]

[0168]

[0169] The purpose of the encoder is to extract high-level semantic information features. In order to enhance the reusability of module features, skip connections are used inside the encoding module. The structure of the encoding module consists of three structures: convolutional layer, batch normalization and ReLU activation layer. This combination enhances the generalization ability of the network. The output feature En of the encoding module at the bth level is b Expressed as

[0170]

[0171] In the formula, b is the extracted b-th layer feature, Encoder b () is the encoder function of the b-th layer, En b-1 is the output feature of the previous layer of level b, En b is the output feature of the encoding module at the bth level, and e is the number of extracted feature layers; the advantage of the encoding module is that the Demodex features flow directly from the bottom module to the top module through the feature extraction function, and the skip connection within the module makes up for the loss of the pooling and convolution layers.

[0172] When the encoding module b=4, the feature compression matrix of the hidden layer is obtained, and the sequence prediction module is directly connected to the classic Transformer model according to the image position behavior index. The self-attention mechanism is used to globally model the features, further improving the model's ability to identify and locate significant Demodex features in the image. This combination not only retains the advantages of convolutional neural networks in feature extraction, but also gives play to the strengths of the Transformer model in global modeling, achieving complementary advantages and significantly improving the performance of the entire model.

[0173] The purpose of the decoding module is to restore the predicted significant demodex region map to the input image size. The number of decoding modules is the same as that of encoding modules. The feature map output by the decoding module is restored to the input image size. The parameter configuration of the encoding module and the decoding module is shown in Table 2. In order to further compensate for the information lost in the downsampling operation of the encoding module, not only the skip connection within the encoding module is used, but also the skip connection is added between the encoding module and the decoding module at the same level. Figure 3 As shown, the output of the encoding module is directly connected to the decoding module at the same level next to it, and the output feature De of the b-th decoding module is b , the calculation formula is expressed as:

[0174]

[0175] In the formula, Decoder b () is the decoder module function of the b-th layer, De b Output feature of the decoding module at level b, Transformer output feature Trans.

[0176] In summary, there are two feature reuse strategies: skip connections between encoder modules and decoder modules, and skip connections within encoder modules; these two strategies reuse the extracted features, help recover the lost information, and reduce the number of model parameters.

[0177] Table 2 Input and output feature map sizes of each module

[0178]

[0179]

[0180] The predicted mite image with the same resolution as the original input image is obtained through the content module, which is conducive to the calculation of the label loss value by comparing with the true value image and providing a numerical basis for gradient back propagation. The specific formula is as follows:

[0181] F c =Content(Decoder(Transformer(Encoder(F i ))))

[0182] Among them, Content() represents the content module function, Decoder() represents the decoder module function, Transformer() represents the Transformer module function, Encoder() represents the encoder module function, and F c is the output feature of the content module.

[0183] The content module consists of two fully convolutional layers. Table 3 shows the specific parameter configuration.

[0184] Table 3 Configuration parameters of content module

[0185] Layer Number of input channels Number of output channels Convolution kernel size Step Length filling Full-Conv 64 32 3 2 - ReLU - - - - - Conv 32 32 3 1 - ReLU - - - - - Full-Conv 32 1 2 1 1

[0186] The hierarchical feature extraction module extracts features by adding the elements of the encoding module and decoding module at the same level, and further summarizes the Demodex features learned by the encoder module and decoder module at each level. The extraction module is composed of cascaded 3 convolutional layers with different kernel sizes, and the configuration is shown in Table 4. The output feature map S of the bth extraction module b The calculation formula is expressed as:

[0187] S b =Extract(En b-1 +De b ),

[0188] In the formula, Extract() is the function of the extraction module, which consists of 1 convolution layer, S b It is the characteristic image after inverting the effective Demodex area.

[0189] Table 4 Configuration parameters of the extraction module

[0190] Extraction module Number of input channels Number of output channels Convolution kernel size E0 1 1 3 E1 64 64 3 E2 64 64 5 E3 128 128 5 E4 256 256 5 E5 512 512 7

[0191] The face area is divided into three parts: effective Demodex infection area, Demodex edge area and uninfected area; by reversing the effective mite area feature connection operation module, the reversal of the effective mite area feature connection process is as follows Figure 4 As shown, first, the prediction graph S of the extraction module is extracted. b+1 Permuting the zero and one values ​​in the , so that the valid area is filled with zeros; then, the predicted demodex infection area map with flipped zeros and ones is element-wise multiplied with the input of the next extraction module to exclude the effective area of ​​Demodex; at this time, in addition to the effective area of ​​Demodex, the output features still include the edge area of ​​Demodex and the area that has not been infected; finally, the extraction module and the prediction map S b+1 Connect and extract the edge features of Demodex, that is, combine the edge features and the previously predicted Demodex area to perform element addition Therefore, the reverse accumulation operation allows the effective object area of ​​deep learning to be accumulated back. Mathematically, the input En of the extraction module b-1 +De b , and the output S of the previous layer b+1 Calculate the characteristic map S after inverting the effective Demodex area b , the calculation formula is expressed as:

[0192]

[0193] The above process calculates the predicted demodex region of the extraction module in the opposite direction from the forward connection subnet from top to bottom. The benefit of reversing the effective demodex region is that the extraction module learns from the smallest feature space, which avoids relearning from the effective region containing the demodex location, the location features of the demodex have been fully extracted in the forward connection subnet; in addition, the reversal of the effective demodex region operation allows the features to continue to propagate in the neural network without affecting the position of the object.

[0194] In addition, reusing the inverted effective Demodex region helps to enhance the features. The sampling factor of the skip connection operation adopted is 2 times, and the upsampling is performed from S b Get twice the size Feature map; mathematically, the output of the extraction module at layer b New features are obtained after skip connection The calculation formula is expressed as:

[0195]

[0196] In the formula, is the weight of the jump connection from extraction module m to b (b>m), where m is 3, 4, or 5 in the formula; is the b-th level output feature map obtained by upsampling, It is the feature map after the jump connection operation of the inverted effective area. In order to make full use of the features extracted by each module, the feature fusion module is used to merge the extraction modules at different levels to obtain the fusion map F fuse , the calculation formula is expressed as:

[0197]

[0198] In the formula, the encoders and decoders at different levels are inverted and the feature fusion is performed to obtain the feature map and the output feature F of the content module c , Concate cascade stacking, Concate() is the cascade feature map matrix along the channel dimension, and Mean() calculates the average value of the elements along the channel dimension.

[0199] Construct the saliency map loss function module: Use the Adam optimizer to train the neural network; in order to calculate the gradient of the neural network back propagation, introduce the total model loss consisting of content loss and extraction loss, corresponding to the forward connection subnet and the backward connection subnet respectively. The model parameters θ are composed of weights W1,…,W L and biases b1,…,b L The binary cross entropy loss σ(θ) is used to evaluate the difference between the predicted saliency map and the true value map, and the gradient adjustment parameters of the model are calculated by back propagation. The binary cross entropy loss calculation formula is expressed as:

[0200] σ(GT,·|θ)=Ylog H L +(1-Y)log(1-H L ), Y∈{0,1},

[0201] Where σ(GT,·|θ) is the distance between the true image GT and the predicted image, and H L The probability that sample L is predicted as a positive class, Y is the sign function, which takes 1 when sample L is a positive class and 0 otherwise, and θ is the model parameter to be adjusted.

[0202] After the training is completed, we need to evaluate the performance of the model on the validation set and the test set, which helps us understand the generalization ability of the model on new data, identify areas that need improvement, optimize and evaluate the model, and ensure that it has good performance and robustness. After the training is completed, for the face RGB images that have not been manually labeled, the regional detection results of the face demodex mites in the image to be tested are obtained; the mean absolute error (MAE) and the maximum F-measure (Max-F) are used to quantitatively evaluate the effect of the proposed model. The quantitative evaluation method is the existing technology in this field. The above two examples are given below, which can be replaced by other quantitative evaluation methods.

[0203] Based on the evaluation results, the model may need to be fine-tuned, which may include adjusting algorithm parameters, optimizing network structure, or collecting more training data. This process is repeated several times until the performance of the model reaches a satisfactory level. Finally, the hyperparameters are adjusted to the model using the optimizer Adam, and the optimizer hyperparameters β = [0.9, 0.999], ∈ = 10 -8 ; During the training phase, the initial learning rate is set to 1e -4 , the batch size is 32, and the unified input image size is 256×256; the first 5 encoding modules of the network are initialized by the weights of the ResNet34 pre-trained model trained on the ImageNet dataset, and the remaining modules use random initialization weights from a normal distribution of (0, 0.01), and the initial biases are all set to 0; when the training accuracy remains unchanged for 40 iteration batches, stop training the network.

[0204] like Figure 6 As shown, based on the model's prediction of facial Demodex regional evaluation and multiple model adjustments, dedicated doctors provide patients with appropriate personalized treatment plans by analyzing the correlation with other physiological indicators of the patients. This detection method for facial Demodex provides dedicated doctors with a strong auxiliary role in diagnosis and treatment.

[0205] When the model performs well, we can deploy it to real applications, such as Figure 7 As shown, this involves integrating the model into a larger system, which allows users to self-check the facial skin condition or observe the skin condition from their mobile phones. The facial skin condition will be photographed using the user's mobile phone or a camera, and after processing and analysis by professional medical equipment, a professional report will be provided to the doctor. The doctor will then analyze the professional report and give professional medical advice and treatment plans.

[0206] After the model is deployed, it is critical to continuously monitor its performance and accuracy. In order to maintain the effectiveness of the model's predictions, new data needs to be collected regularly for testing and retraining the model.

[0207] The application on mobile devices has the characteristics of lightweight model, optimized model structure, and reduced computational complexity, so that it can run efficiently on mobile devices; in terms of user interface design, a user's mobile application can be developed to allow users to take and upload skin pictures and obtain mite detection results, making the detection method more convenient.

[0208] Comparative Example 1

[0209] In order to make the purpose, technical scheme and advantages of the present invention clearer, the data set collected from users treated with Demodex mites is used as a comparative example to further describe the process of the present invention in detail. Obviously, the comparative example described is only a part of the comparative examples of the present invention, rather than all the comparative examples.

[0210] In this comparative example, the PyTorch code library is used to implement the model, and a general face mite area detection system is developed. The user privacy authorization and network public image data set are adopted, and the training set, test set and expert manual labeling labels of each sample have been divided. At the same time, the present invention also conducts comparative experiments with RAS, DSS, ResNet and other current general target detection methods with good effects. The results of face demodex area detection are shown in Table 5. According to the table, it can be found that the method of the present invention has better performance in face demodex area detection, and the results of the visualized part of the samples are almost close to the expert labeling, such as Figure 5 As shown, the effectiveness and stability of the method for detecting Demodex patient areas are demonstrated.

[0211] Table 5 Experimental results of model prediction of Demodex area on human face

[0212] method Mean absolute error Maximum F measurement value ResNet 0.802 0.690 DSS 0.416 0.775 RAS 0.045 0.801 Model of the present invention 0.037 0.861

[0213] according to Figure 5 From the comparison, we can conclude that although the detection range of the model of the present invention is expanded compared with the common annotated area of ​​5 experts and includes additional infected areas with semi-transparent confidence, this does not mean that there is an error in the detection result of the model; in fact, these semi-transparent areas do show a certain degree of mild infection in the original input face image; although these areas are not annotated by all experts at the same time, their existence cannot be ignored. This finding is due to several factors:

[0214] First, there may be subjective differences among different experts during the labeling process, resulting in certain deviations in the judgment of the degree of infection.

[0215] Secondly, the model proposed in this invention uses deep learning and image processing technology of bidirectional feature propagation, which can more accurately capture subtle changes in the skin of the face, including those mildly infected areas that may be overlooked by experts. From the perspectives of morphological feature diversity patterns, background and target distinction patterns, density and aggregation patterns, dynamic change patterns, and coexisting pathological condition patterns, this invention supplements and improves the areas annotated by experts. Figure 5 As shown, specifically:

[0216] 1. In terms of morphological feature diversity patterns (the first and second rows), compared with expert annotations, the model additionally detected Demodex mites on both sides of the cheeks and the forehead, which improved the comprehensiveness of the detection. Compared with the prior art, the present invention has an outstanding detection effect and a significant improvement in Demodex detection technology;

[0217] 2. In terms of density and clustering patterns, the detection results in the third row use a semi-transparent representation method to highlight the characteristics of high-density mite areas;

[0218] 3. In terms of excluding the coexisting pathological condition pattern (the fourth row), the model can effectively distinguish the patient's coexisting skin erythema pathology and exclude it from the Demodex detection area;

[0219] 4. In terms of distinguishing dynamic change patterns (fifth row), the model can better distinguish the acute phase and remission phase of patients, which is helpful for clinical evaluation;

[0220] 5. In terms of the background and target distinction mode (the sixth row), the model can accurately distinguish the yellow skin of the patient's facial background from the target Demodex area, especially in the lateral area of ​​the nose. In addition, the model also reveals the speckle distribution of Demodex in the lateral cheek and upper lip area. It is worth noting that although the expert annotation can clearly distinguish the area with the highest density of Demodex, it ignores the medium density of Demodex (closer to the pink area).

[0221] Finally, the semi-transparent confidence setting of the model of the present invention also provides more flexibility for its detection results. By adjusting the confidence threshold, we can balance the sensitivity and specificity of the detection results according to actual needs, so as to better meet the needs of different application scenarios.

[0222] The above description is a detailed description of the preferred feasible embodiments and comparative examples of the present application, but the embodiments and comparative examples are not intended to limit the patent application scope of the present application. All equivalent changes or modified changes made under the technical spirit suggested by the present application should fall within the patent scope covered by the present application.

Claims

1. A method for detecting the Demodex area on a human face, characterized in that: The specific steps include: Step 1: Use the image acquisition module to obtain multiple images of Demodex mites on the human face; Step 2: Annotate the Demodex regions in all facial Demodex images to construct a facial Demodex image dataset; Step 3: Build a deep neural network model to capture the complex patterns of facial Demodex in image data; Step 4: The labeled color image of human face demodex mites is used to train the deep neural network model, and the difference between the predicted value and the true value image is compared through the constructed loss function. The back propagation algorithm is used to iteratively calculate and adjust the model's own weights to improve the accuracy of predicting the human face demodex mites area; Step 5: Use the trained deep neural network model to process the face image that has not been manually labeled to obtain the face demodex area detection result of the image to be tested; Step 6: Output the result of the facial Demodex region detection of the image to be tested through the output module.

2. The method for detecting the Demodex area on the human face according to claim 1, characterized in that: Complex patterns include patterns of morphological feature diversity, background and target distinction, density and aggregation, dynamic change, and coexisting pathological conditions.

3. The method for detecting Demodex areas on the human face according to claim 2, characterized in that: In step 1, multiple facial Demodex images are processed by a preprocessing module; after the annotation is completed in step 2, multiple truth maps are generated; in step 3, the deep neural network model is a deep neural network model with a Transformer bidirectional skip connection, and the complex pattern of facial Demodex in the image data in step 3 is captured by a significant Demodex detection module composed of this deep neural network model; in step 5, the trained Transformer bidirectional skip connection deep neural network model is used to process the facial color images that have not been manually labeled.

4. The method for detecting the Demodex area on the human face according to claim 3, characterized in that: The specific steps of constructing a Transformer bidirectional skip connection deep neural network model in step 3 are: Step a: using multiple labeled facial Demodex images, and converting the feature maps obtained by the input module into a feature matrix; Step b: The encoder module extracts high-order mite region features layer by layer, the feature map size gradually decreases, and the spatial dimension gradually increases, obtaining a high-dimensional low-rank compressed feature matrix; Step c: The output feature map of the encoder module is passed through the sequence prediction module to predict the spatial sequence position feature matrix of Demodex; Step d: The spatial sequence position feature matrix is ​​restored to the feature map size layer by layer through the decoder module, and then passed through the content module to obtain a decoded feature map of the same size as the feature map of the input module; Step e: extracting the features of the decoder module process layer by layer in the decoder module, and performing the inversion connection operation with the feature map of the same layer of the encoder module in the inversion effective mite area feature connection module, and obtaining the extracted feature maps of different scales through the layered extraction feature module; Step f: After the extracted feature maps of different scales are stacked with the decoded feature maps output by the content module, they are fused into feature maps of uniform size through a feature fusion module to obtain a predicted feature map; Step g: The predicted feature map is outputted by the feature fusion module to obtain the significant area of ​​the mite area, and the deep neural network model is used to form a significant demodex detection module to capture the complex patterns in the facial demodex image data.

5. The method for detecting the Demodex area on the human face according to claim 4, characterized in that: In step a, the specific formula for obtaining the feature map through the input module is as follows: F i =Max(Conv(I input )) In the formula, I input is the input image, F i is the feature map output by the i-th layer, Max is the maximum pooling layer function, and Conv is the convolution function of the convolution layer; In step b, the specific formula for extracting high-order mite area features layer by layer by the encoder module is as follows: In the formula, b is the extracted b-th layer feature, Encoder b ( ) is the encoder function of the b-th layer, En b-1 is the output feature of the previous layer of level b, En b is the output feature of the encoding module at level b, and e is the number of extracted feature layers; In step c, the output feature graph of the encoder module is passed through the sequence prediction module to obtain the specific formula for predicting the spatial sequence position feature matrix of Demodex mites as follows: Trans=Transformer(A b ) Where Trans is the spatial sequence position feature matrix, Transformer() is the Transformer model function; In the step e, the high-order features of Demodex and the spatial sequence position feature matrix are decoded layer by layer in the decoder module, and the high-dimensional low-rank features are gradually restored to the low-dimensional Demodex significant area. The specific formula is as follows: In the formula, Decoder b () is the decoder module function of the b-th layer, De b Output features for the decoding module at level b; In step e, the specific calculation formula for performing the inversion operation in the inversion effective mite area feature connection module is as follows: In the formula, Extract() is the function of the extraction module, which consists of l convolutional layers, S b This is the feature map after inverting the effective Demodex area. is the element-by-element multiplication of the feature matrix, Add element-wise to the feature matrix; In step f, the specific formula for the decoded feature map of the same size as the feature map of the input module obtained through the content module is as follows: F c =Content(Decoder(Transformer(Encoder(F i )))) Among them, Content() represents the content module function, Decoder() represents the decoder module function, Transformer() represents the Transformer module function, Encoder() represents the encoder module function, and F c is the output feature of the content module; In step f, the extracted feature maps of different scales are fused into a feature map of uniform size through a feature fusion module, and are stacked with the decoded feature map output by the content module. The mean of the dimensions is taken, and the specific calculation formula for the predicted feature map is obtained by using the mean strategy as follows: In the formula, the feature map is obtained by feature fusion after the encoder module and the decoder module are inverted. F c is the output feature of the content module, Concate() is the cascade stacking function, Mean() is the average value of the elements along the channel dimension, and F fuse The obtained feature fusion map.

6. The method for detecting the Demodex area on the human face according to claim 1, characterized in that: In the step 4, the saliency map loss function is constructed and the Adam optimizer is used to train the neural network; the back propagation algorithm is used for iterative calculation, and the gradient of the back propagation of the neural network is first calculated, and the total model loss composed of the content loss and the extraction loss is introduced, which correspond to the forward connection subnet and the reverse connection subnet respectively; the model parameters θ include weights W1, ..., W L and biases b1,…,b L ; The binary cross entropy loss σ(θ) is used to evaluate the difference between the predicted saliency map and the true value map, and the gradient adjustment parameters of the back propagation calculation model are expressed as follows: σ(GT,·|θ)=Ylog H L +(1-Y)log(1-H L ),Y∈{0,1} Where σ(GT,·|θ) is the distance between the true image GT and the predicted image, and H L The probability that sample L is predicted as a positive class, Y is the sign function, which takes 1 when sample L is a positive class and 0 otherwise, and θ is the model parameter to be adjusted.

7. A detection device for the Demodex area on the human face, characterized in that: include: An image acquisition module, used for acquiring color images of Demodex mites on the human face; A preprocessing module is used to process the color image of Demodex mites on the human face by resizing the image, normalizing the pixel values, and cropping and flipping the image; The significant demodex detection module is used to capture the complex patterns in the image data to be tested and calculate the regional detection results of demodex mites on the human face in the image to be tested; The output module is used to output the extracted Demodex area on the human face.

8. The detection device for the Demodex area on the human face according to claim 7, characterized in that: The significant Demodex detection module comprises: The input module, including multiple convolutional layers and maximum pooling, is used to preprocess the input original human face demodex mite image; in this process, the resolution of input and output will be consistent with the original image to ensure the quality and clarity of the image; The encoder module includes multiple encoders and is composed of multiple convolutional layers, batch normalization, and ReLU activation layers. It is used to extract the characteristic functions of Demodex mites on the face and gradually obtain high-order semantic features. The multi-layer convolutional layers can extract various features and patterns of Demodex mites from the image. The decoder module includes multiple decoders and is composed of multiple layers of deconvolution, which is used for low-order demodex information reconstruction and pixel-level binary classification. The high-level semantic features extracted by the encoding module are gradually restored back to the original face infection image, and the image area is classified at the same time. The deconvolution layer maps the high-level features back to the original image space to restore the details and structure of the image. The sequence prediction module, which consists of Transformer modules, is used to perform facial region sequence encoding on the compressed feature matrix output by the encoder model, and treats each row of Demodex features as a spatial sequence. Through sequence analysis of the feature matrix, the sequence prediction module captures the spatial variation and trend of Demodex in different facial regions, thereby accurately predicting the number and distribution of Demodex. Content module, a module for post-processing the output predicted image, with the input and output resolutions being consistent with the original image; this module performs post-processing on the output predicted human face demodex mite image, including color correction, denoising and sharpening; A reverse effective mite region feature connection module is used to perform a reverse connection operation between the encoder module and the decoder module; A hierarchical feature extraction module is used to extract the encoder module and the decoder module of the same level of the module to further extract significant Demodex features; The feature fusion module adopts the mean strategy to merge the output Demodex feature maps at different levels to obtain the output results.

9. A system for detecting the Demodex area on the human face, characterized in that: The invention comprises a server and a client, wherein the server comprises a processor and a memory; the memory stores facial demodex image data, program codes and computer executable instructions, and when executed by the processor, is used to implement any one of the facial demodex area detection methods according to claims 1 to 6; the client is used to upload images and display results.

10. A detection storage medium for the Demodex region of a human face, characterized in that: Used to store facial Demodex image data, program codes and computer executable instructions on a computer, and process facial Demodex image data by running the program code stored on the computer to implement the facial Demodex area detection method described in any one of claims 1 to 6.

Citation Information

Cited By

  • Intelligent control method and device for multifunctional use of hair removal instrument based on user identification

    CN120267391A

  • Night scene lane line detection method

    CN121564677A

  • A night scene lane line detection method

    CN121564677B