Skin disease feature recognition method and system based on image processing

Abstract

The invention relates to the technical field of medical artificial intelligence and computer vision, in particular to a skin disease feature recognition method based on image processing, and the method comprises the following steps: S1, obtaining multi-modal data of a skin lesion region, including an RGB image, a thermal imaging image and a patient medical history text; s2, performing illumination correction, noise suppression and lesion region segmentation on the image data; the skin disease feature recognition system based on image processing comprises the following contents: a multispectral image acquisition module supporting synchronous acquisition of visible light, infrared and depth data; in the using process of the method and system, the diagnosis efficiency and accuracy of medical staff can be improved, the medical staff can be effectively assisted to analyze and confirm the type of the illness state, and meanwhile the doctor seeing efficiency of the medical staff can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of medical artificial intelligence and computer vision technology, and more specifically, to a method and system for identifying skin disease features based on image processing. Background Art

[0002] Skin diseases refer to diseases that occur in the skin, mucous membranes and their appendages, and are caused by a variety of reasons, such as physical and chemical factors, pathogen infection, endocrine and metabolic factors, immunogenicity and degenerative changes.

[0003] In practice, the diagnosis of skin diseases is generally based on doctors' experience. Traditional diagnosis relies on doctors' subjective experience, has a high misdiagnosis rate, and is particularly weak in distinguishing complex skin diseases (such as melanoma and psoriasis). Even if medical artificial intelligence is used to diagnose the condition, the existing single-modality data (such as RGB images) is difficult to fully characterize the lesion characteristics (such as subcutaneous inflammation and texture gradients). At the same time, the existing algorithms are sensitive to uneven lighting and noise interference, and lack dynamic data fusion.

[0004] Therefore, we proposed a skin disease feature recognition method and system based on image processing to solve the above problems. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a method and system for skin disease feature recognition based on image processing to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for identifying skin disease features based on image processing, comprising the following methods:

[0007] Step S1: Acquire multimodal data of the skin lesion area, including RGB images, thermal imaging images, and patient medical history text;

[0008] Step S2: performing illumination correction, noise suppression, and lesion area segmentation on the image data;

[0009] Step S3: Extract deep features of RGB images and thermal images through a dual-branch convolutional network, and combine it with the BERT model to extract text semantic features;

[0010] Step S4: Use the attention mechanism to dynamically fuse multimodal features and generate a joint feature vector;

[0011] Step S5: Output the disease type, confidence level, and suspected malignant area label based on the integrated learning model.

[0012] In a preferred embodiment, in a dual-branch convolutional network:

[0013] The RGB image branch adopts an improved U-Net structure and integrates the feature pyramid network to achieve multi-scale feature extraction.

[0014] In a preferred embodiment, the thermal imaging branch uses a lightweight depth-wise separable convolutional network to output an abnormal temperature distribution feature map.

[0015] In a preferred embodiment, in the multimodal fusion module, the weight distribution of image features and text features is achieved through a cross-attention mechanism, and the formula is:

[0016]

[0017] Among them, Q is the image feature matrix, K is the text feature matrix, and d is the feature dimension.

[0018] In a preferred embodiment, the ensemble learning model includes a classification model, and the classification model adopts a dynamic weighting strategy, adjusts the classification threshold according to the disease prevalence, and optimizes the decision boundary in combination with clinical prior knowledge;

[0019] The classification and recognition module includes an anomaly detection submodule, which triggers a manual review prompt when the confidence level is lower than the threshold.

[0020] The skin disease feature recognition system based on image processing includes the following:

[0021] Multispectral image acquisition module: supports simultaneous acquisition of visible light, infrared and depth data;

[0022] Preprocessing module: performs adaptive illumination correction and lesion area segmentation;

[0023] Multi-scale feature extraction module: Integrates CNN and graph neural network to analyze the relationship between local texture and global topology;

[0024] User interaction module: provides visual reports and supports user feedback and corrections.

[0025] In a preferred embodiment, the pre-processing module uses an adaptive threshold segmentation algorithm combined with edge gradient information of the lesion area to achieve precise positioning.

[0026] In a preferred embodiment, the user interaction module includes an explainable AI component that highlights key lesion areas that affect the classification results through heat maps.

[0027] In a preferred embodiment, the system is deployed on an edge computing terminal, supports online model updates and incremental learning, and the optimized model version is synchronized to the cloud server.

[0028] In a preferred embodiment, in the multi-scale feature extraction module:

[0029] An improved convolutional neural network (CNN) is used to extract local texture and color features;

[0030] Combined with graph neural network (GNN) to analyze the correlation characteristics between the lesion area and the surrounding skin;

[0031] Fuse patient medical history data for multimodal feature enhancement.

[0032] The technical effects and advantages of the present invention are as follows:

[0033] During use, this method and system can improve the diagnostic efficiency and accuracy of medical staff, and can effectively assist medical staff in analyzing and confirming the type of disease. At the same time, it can also effectively improve the efficiency of medical staff in seeing patients, and can deal with complex conditions, meeting the actual use needs of medical staff, and indirectly improving the practicality of this method and system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of the process of the skin disease feature recognition method based on image processing in the present invention. DETAILED DESCRIPTION

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0036] Reference Figure 1 , the skin disease feature recognition method based on image processing includes the following methods:

[0037] Step S1: Acquire multimodal data of the skin lesion area, including RGB images, thermal imaging images, and patient medical history text;

[0038] Step S2: performing illumination correction, noise suppression, and lesion area segmentation on the image data;

[0039] Step S3: Extract deep features of RGB images and thermal images through a two-branch convolutional network, and extract text semantic features in combination with the BERT model; in the two-branch convolutional network:

[0040] The RGB image branch adopts an improved U-Net structure and integrates a feature pyramid network to achieve multi-scale feature extraction;

[0041] The thermal imaging branch uses a lightweight deep separable convolutional network to output abnormal temperature distribution feature maps;

[0042] Step S4: Use the attention mechanism to dynamically fuse multimodal features and generate a joint feature vector;

[0043] In the multimodal fusion module, the weight distribution of image features and text features is achieved through the cross-attention mechanism, and the formula is:

[0044]

[0045] Among them, Q is the image feature matrix, K is the text feature matrix, and d is the feature dimension;

[0046] Step S5: Outputting the disease type, confidence level, and suspected malignant area label based on the ensemble learning model; the ensemble learning model includes a classification model, and the classification model adopts a dynamic weighting strategy to adjust the classification threshold according to the disease prevalence and optimize the decision boundary in combination with clinical prior knowledge;

[0047] The classification and recognition module includes an anomaly detection submodule, which triggers a manual review prompt when the confidence level is lower than the threshold.

[0048] The skin disease feature recognition system based on image processing is characterized by: including the following contents:

[0049] Multispectral image acquisition module: supports simultaneous acquisition of visible light, infrared and depth data;

[0050] Preprocessing module: performs adaptive illumination correction and lesion area segmentation; the preprocessing module uses an adaptive threshold segmentation algorithm and combines the edge gradient information of the lesion area to achieve accurate positioning;

[0051] Multi-scale feature extraction module: Integrates CNN and graph neural network to analyze the relationship between local texture and global topology; In the multi-scale feature extraction module:

[0052] An improved convolutional neural network (CNN) is used to extract local texture and color features;

[0053] Combined with graph neural network (GNN) to analyze the correlation characteristics between the lesion area and the surrounding skin;

[0054] Fuse patient medical history data for multimodal feature enhancement;

[0055] User interaction module: Provides visual reports and supports user feedback and corrections; the user interaction module includes an explainable AI component that highlights key lesion areas that affect classification results through heat maps.

[0056] The system is deployed on edge computing terminals, supports online model updates and incremental learning, and the optimized model version is synchronized to the cloud server.

[0057] Example 1:

[0058] 1. Data Collection and Preprocessing

[0059] The input RGB images (size 512×512) and thermal imaging images (256×256) are collected by a mobile device integrated with a multispectral camera (wavelength range 400-1500nm).

[0060] Preprocessing: RGB images were subjected to denoising (Gaussian filtering with σ = 1.5) and CLAHE enhancement (grid size 8 × 8), and thermal imaging images were normalized.

[0061] 2. Feature extraction and fusion

[0062] Dual-branch network processing:

[0063] RGB branch: Global features are extracted through ResNet-50, and the SLIC algorithm (number of superpixels = 200) is used to divide the local area and calculate the HOG features (number of directions = 9).

[0064] Thermal imaging branch: Use the U-Net model to segment the lesion area and remove background interference.

[0065] Feature fusion: Multi-scale features are fused through the feature pyramid network (FPN), and combined with GNN to construct a topological relationship map between the lesion area and healthy skin.

[0066] 3. Classification and result output

[0067] The classification module uses a pre-trained XGBoost model (the training data contains 100,000 labeled skin images) to output a diagnosis result (such as "psoriasis", with a confidence level of 92%).

[0068] Supports users to manually adjust boundary segmentation deviation and trigger incremental model training to optimize performance.

[0069] 4. Test Results

[0070] Dataset: ISIC 2020 and self-built clinical dataset (covering 10 common skin diseases).

[0071] Performance indicators: average recognition accuracy 95.2%, sensitivity 92.4%, specificity 89.7%, and AUC value 0.94.

[0072] Example 2: Decision Optimization Method Combined with Medical History Text

[0073] 1. Multimodal Input

[0074] Input clinical images (RGB or thermal imaging) and text-based medical history data (such as "the patient has had the disease for 2 years, accompanied by itching").

[0075] 2. Feature fusion and classification correction

[0076] Text feature extraction: Encode medical history text through the Transformer model.

[0077] Multimodal fusion: Combine text features with image features (extracted by ResNet-50) and use the cross-attention mechanism to optimize feature association.

[0078] Classification correction: Output the correction result (such as "atopic dermatitis", confidence level 96%).

[0079] 3. High-risk markers (for malignant melanoma)

[0080] Added calculation of local feature asymmetry (based on principal component analysis). If the asymmetry index is greater than 0.7, it is marked as high risk.

[0081] Hardware configuration and algorithm flow

[0082] 1. Hardware Configuration

[0083] Image acquisition end: mobile device with integrated multispectral camera.

[0084] Server side: Deploy a GPU-accelerated deep learning inference engine (supporting online model updates).

[0085] 2. Algorithm Process

[0086] Step 1: U-Net segments the lesion area and ResNet-50 extracts basic features.

[0087] Step 2: GNN constructs a topological relationship graph, and FPN fuses multi-scale features.

[0088] Step 3: The Transformer model fuses image and text features, and XGBoost outputs the classification results.

[0089] The above implementation method can be adapted to different application scenarios by adjusting model parameters (such as using AdamW as the optimizer, learning rate = 3e-5, weight decay = 0.05) or data enhancement strategy (random rotation ±30°, brightness jitter ±20%).

[0090] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying skin disease features based on image processing, characterized in that: Includes the following methods: Step S1: Acquire multimodal data of the skin lesion area, including RGB images, thermal imaging images, and patient medical history text; Step S2: performing illumination correction, noise suppression, and lesion area segmentation on the image data; Step S3: Extract deep features of RGB images and thermal images through a dual-branch convolutional network, and combine it with the BERT model to extract text semantic features; Step S4: Use the attention mechanism to dynamically fuse multimodal features and generate a joint feature vector; Step S5: Output the disease type, confidence level, and suspected malignant area label based on the integrated learning model.

2. The method for identifying skin disease features based on image processing according to claim 1, characterized in that: In a two-branch convolutional network: The RGB image branch adopts an improved U-Net structure and integrates the feature pyramid network to achieve multi-scale feature extraction.

3. The method for identifying skin disease features based on image processing according to claim 2, characterized in that: The thermal imaging branch uses a lightweight depth-wise separable convolutional network to output abnormal temperature distribution feature maps.

4. The method for identifying skin disease features based on image processing according to claim 1, wherein: In the multimodal fusion module, the weight distribution of image features and text features is achieved through the cross-attention mechanism, and the formula is: Among them, Q is the image feature matrix, K is the text feature matrix, and d is the feature dimension.

5. The method for identifying skin disease features based on image processing according to claim 1, wherein: The ensemble learning model includes a classification model, which uses a dynamic weighting strategy to adjust the classification threshold according to the disease prevalence and optimize the decision boundary based on clinical prior knowledge. The classification and recognition module includes an anomaly detection submodule, which triggers a manual review prompt when the confidence level is lower than the threshold.

6. A skin disease feature recognition system based on image processing, characterized by: Includes the following: Multispectral image acquisition module: supports simultaneous acquisition of visible light, infrared and depth data; Preprocessing module: performs adaptive illumination correction and lesion area segmentation; Multi-scale feature extraction module: Integrates CNN and graph neural network to analyze the relationship between local texture and global topology; User interaction module: provides visual reports and supports user feedback and corrections.

7. The skin disease feature recognition system based on image processing according to claim 6, characterized in that: The preprocessing module uses an adaptive threshold segmentation algorithm and combines the edge gradient information of the lesion area to achieve accurate positioning.

8. The method and system for skin disease feature recognition based on image processing according to claim 6, characterized in that: The user interaction module includes an explainable AI component that highlights key lesion areas that affect the classification results through heat maps.

9. The method and system for identifying skin disease features based on image processing according to claim 6, characterized in that: The system is deployed on edge computing terminals, supports online model updates and incremental learning, and the optimized model version is synchronized to the cloud server.

10. The method and system for identifying skin disease features based on image processing according to claim 6, characterized in that: In the multi-scale feature extraction module: An improved convolutional neural network (CNN) is used to extract local texture and color features; Combined with graph neural network (GNN) to analyze the correlation characteristics between the lesion area and the surrounding skin; Fuse patient medical history data for multimodal feature enhancement.

Images