Feature extraction and recognition method for skin lesion hyperspectral image classification

Abstract

The invention discloses a feature extraction and recognition method for skin lesion hyperspectral image classification, and belongs to the technical field of image processing, and the method comprises the following steps: S1, carrying out the preprocessing of a hyperspectral image; s2, extracting diversity features of the preprocessed hyperspectral image; s3, the extracted features enter a feature fusion module, and feature fusion is realized through a three-dimensional full convolutional neural network of multi-granularity feature fusion; s4, after feature fusion, lesion region segmentation is carried out in a segmentation module; s5, classifying the segmented features in a classification module; s6, enabling the classified lesion areas to enter a detection module, and improving the detection precision and real-time performance of lesion cells in the microscopic hyperspectral image; and S7, verifying a detection result of the detection module by a clinician according to a pathological result of a pathological expert. The feature extraction and recognition method for skin lesion hyperspectral image classification has higher diagnosis and classification capabilities, and is suitable for various medical image analysis tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a feature extraction and recognition method for classification of skin lesion hyperspectral images. Background Art

[0002] With the increasing incidence of skin diseases, the demand for automated detection of skin diseases is also increasing. In this regard, machine learning methods and deep learning methods have become the current research hotspots. Among them, machine learning methods mainly use traditional support vector machines, decision trees, random forests and other machine learning algorithms. The algorithm has fast calculation speed and strong model interpretability, but it performs poorly when processing complex data, making it difficult to accurately classify and diagnose skin disease images. Summary of the invention

[0003] The purpose of the present invention is to provide a feature extraction and recognition method for classification of hyperspectral images of skin lesions, improve the detection accuracy and real-time performance of diseased cells in microscopic hyperspectral images, have higher diagnosis and classification capabilities, and be suitable for a variety of medical image analysis tasks.

[0004] To achieve the above object, the present invention provides a feature extraction and recognition method for skin lesion hyperspectral image classification, comprising the following steps:

[0005] S1, input the hyperspectral image into the data processing module for preprocessing;

[0006] S2, the pre-processed hyperspectral image is input into the feature extraction module, and the spectral feature and spatial feature extraction based on the three-dimensional convolutional neural network are used to obtain the diversity characteristics of skin lesion tissue from the spectral dimension and the spatial dimension respectively;

[0007] S3, the extracted features enter the feature fusion module, and feature fusion is achieved through a three-dimensional fully convolutional neural network with multi-granularity feature fusion;

[0008] S4, after feature fusion, enters the segmentation module, which segments the lesion area based on a three-dimensional convolutional neural network;

[0009] S5, the segmented features enter the classification module for classification;

[0010] S6. The classified lesion area enters the detection module. Based on the three-dimensional single-stage detection network of the multi-task joint anchor mechanism, the weighted regression fusion method is used to improve the detection accuracy and real-time performance of lesion cells in microscopic hyperspectral images.

[0011] S7. The test results obtained by the detection module are verified by the clinician based on the pathological results obtained from the pathology expert.

[0012] Preferably, the preprocessing in S1 reduces data redundancy and improves image quality by spectral normalization, denoising, and band selection. The band selection is a method based on mutual information to select spectral bands that are highly correlated with pathological tissue characteristics. The calculation formula of mutual information is as follows:

[0013]

[0014] Among them, X and Y represent different bands and pathological tissue characteristics respectively, p(x,y) represents the joint probability distribution, and p(x) and p(y) represent the marginal probability distribution.

[0015] Preferably, the three-dimensional convolutional neural network in S2 includes a multi-channel input layer, a convolution layer, a batch normalization layer, an activation function, an adaptive pooling layer and a fully connected layer.

[0016] Preferably, the step of extracting spectral features in S2 is to extract the spectral features of each pixel in multiple bands using a three-dimensional convolution operation, and the spectral feature extraction formula is as follows:

[0017] S(i,j,k)=(k*I)(i,j,k)=∑ m I(i,j,k)*K(im,jn,kp);

[0018] Among them, i, j, k are three-dimensional vector indices, m, n, p are convolution kernel sizes, I(i, j, k) is the input three-dimensional vector, K(i, j, k) is the three-dimensional convolution kernel, and S(i, j, k) is the extracted three-dimensional feature.

[0019] Preferably, the feature fusion module in S3 is used to fuse spectral features and spatial features to form a multi-dimensional feature representation. First, spectral features and spatial features are extracted through a conventional three-dimensional convolutional neural network, and information from different channels and levels is obtained respectively. Through multi-granularity feature fusion, spectral information and spatial position information of different scales are combined to form a high-dimensional feature representation. At the same time, the features are decoupled based on the mutual information entropy method. The formula for feature fusion is as follows:

[0020] F = Concat(S,C);

[0021] Among them, F represents the fused feature vector, S and C represent the spectral feature and spatial feature respectively.

[0022] Preferably, the segmentation module in S4 is based on a segmentation model of a three-dimensional fully convolutional neural network, and realizes accurate segmentation of the lesion area in the microscopic hyperspectral image through multi-task joint learning. The segmentation model adopts an adaptive loss function to dynamically optimize the segmentation accuracy. The formula of the adaptive loss function is as follows:

[0023]

[0024] Among them, y i and They represent the true label and the predicted label respectively, and α is the weight coefficient of the loss function.

[0025] Preferably, the classification formula in the classification module in S5 is as follows:

[0026] f(x)=softmax(W2*ReLU(W1*x+b1)+b2);

[0027] Among them, x is the input feature vector, W1 and W2 are weight matrices, and b1 and b2 are bias terms.

[0028] Preferably, the detection module in S6 is based on a three-dimensional single-stage detection network with a multi-task joint anchor mechanism, and improves the detection accuracy and real-time performance of diseased cells in microscopic hyperspectral images through a weighted regression fusion method.

[0029] Therefore, the present invention adopts the above-mentioned feature extraction and recognition method for classification of skin lesion hyperspectral images to improve the detection accuracy and real-time performance of diseased cells in microscopic hyperspectral images, has higher diagnosis and classification capabilities, and is suitable for a variety of medical image analysis tasks.

[0030] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of an embodiment of a feature extraction and recognition method for skin lesion hyperspectral image classification of the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0033] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0034] Embodiment 1

[0035] like Figure 1 As shown, the present invention provides a feature extraction and recognition method for skin lesion hyperspectral image classification, comprising the following steps:

[0036] S1, input the hyperspectral image into the data processing module for preprocessing;

[0037] The main purpose of the data processing module is to preprocess the hyperspectral image to reduce noise, improve image quality and optimize feature extraction. Preprocessing reduces data redundancy and improves image quality through spectral normalization, denoising and band selection. Band selection is based on mutual information to select spectral bands that are highly correlated with pathological tissue features; the calculation formula of mutual information is as follows:

[0038]

[0039] Among them, X and Y represent different bands and pathological tissue characteristics respectively, p(x,y) represents the joint probability distribution, and p(x) and p(y) represent the marginal probability distribution.

[0040] S2, the pre-processed hyperspectral image is input into the feature extraction module, and the spectral feature and spatial feature extraction based on the three-dimensional convolutional neural network are used to obtain the diversity characteristics of skin lesion tissue from the spectral dimension and the spatial dimension respectively;

[0041] The three-dimensional convolutional neural network contains a multi-channel input layer, a convolution layer, a batch normalization layer, an activation function, an adaptive pooling layer, and a fully connected layer.

[0042] The step of spectral feature extraction is to use three-dimensional convolution operation to extract the spectral features of each pixel in multiple bands. The spectral feature extraction formula is as follows:

[0043] S(i,j,k)=(k*I)(i,j,k)=∑ m I(i,j,k)*K(im,jn,kp);

[0044] Among them, i, j, k are three-dimensional vector indices, m, n, p are convolution kernel sizes, I(i, j, k) is the input three-dimensional vector, K(i, j, k) is the three-dimensional convolution kernel, and S(i, j, k) is the extracted three-dimensional feature.

[0045] S3, the extracted features enter the feature fusion module, and feature fusion is achieved through a three-dimensional fully convolutional neural network with multi-granularity feature fusion;

[0046] The feature fusion module is used to fuse spectral features and spatial features to form a multi-dimensional feature representation. First, the spectral features and spatial features are extracted through a conventional three-dimensional convolutional neural network to obtain information from different channels and levels. Then, through multi-granularity feature fusion, spectral information and spatial position information of different scales are combined to form a high-dimensional feature representation. At the same time, the method based on mutual information entropy is used to decouple the features and eliminate redundant information, thereby improving classification accuracy. The formula for feature fusion is as follows:

[0047] F = Concat(S,C);

[0048] Among them, F represents the fused feature vector, S and C represent the spectral feature and spatial feature respectively.

[0049] S4, after feature fusion, enters the segmentation module, which segments the lesion area based on a three-dimensional convolutional neural network;

[0050] The segmentation module is based on a segmentation model of a three-dimensional fully convolutional neural network. It achieves accurate segmentation of lesion areas in microscopic hyperspectral images through multi-task joint learning. The segmentation model uses an adaptive loss function to dynamically optimize the segmentation accuracy. The formula of the adaptive loss function is as follows:

[0051]

[0052] Among them, y i and They represent the true label and the predicted label respectively, and α is the weight coefficient of the loss function.

[0053] S5, the segmented features enter the classification module for classification;

[0054] The classification formula in the classification module is as follows:

[0055] f(x)=softmax(W2*ReLU(W1*x+b1)+b2);

[0056] Among them, x is the input feature vector, W1 and W2 are weight matrices, and b1 and b2 are bias terms.

[0057] S6. The classified lesion area enters the detection module. Based on the three-dimensional single-stage detection network of the multi-task joint anchor mechanism, the weighted regression fusion method is used to improve the detection accuracy and real-time performance of lesion cells in microscopic hyperspectral images.

[0058] The detection module is based on a three-dimensional single-stage detection network with a multi-task joint anchor mechanism, and improves the detection accuracy and real-time performance of diseased cells in microscopic hyperspectral images through a weighted regression fusion method. Detection network: The detection network combines a learning strategy guided by multi-channel features to optimize the positioning and classification performance of the detection frame.

[0059] S7. The test results obtained by the detection module are verified by the clinician based on the pathological results obtained from the pathology expert.

[0060] The biopsy tissue block is first made into HE-stained sections, and then one piece is sent to a clinical pathologist for judgment, and the other piece is judged by the method proposed in this patent. The results obtained by the two are compared with each other, and the clinician determines whether the results confirm each other to verify the accuracy of this method.

[0061] Therefore, the present invention adopts the above-mentioned feature extraction and recognition method for classification of skin lesion hyperspectral images to improve the detection accuracy and real-time performance of diseased cells in microscopic hyperspectral images, has higher diagnosis and classification capabilities, and is suitable for a variety of medical image analysis tasks.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A feature extraction and recognition method for classification of hyperspectral images of skin lesions, characterized by: The following steps are involved: S1, input the hyperspectral image into the data processing module for preprocessing; S2, the pre-processed hyperspectral image is input into the feature extraction module, and the spectral feature and spatial feature extraction based on the three-dimensional convolutional neural network are used to obtain the diversity characteristics of skin lesion tissue from the spectral dimension and the spatial dimension respectively; S3, the extracted features enter the feature fusion module, and feature fusion is achieved through a three-dimensional fully convolutional neural network with multi-granularity feature fusion; S4, after feature fusion, enters the segmentation module, which segments the lesion area based on a three-dimensional convolutional neural network; S5, the segmented features enter the classification module for classification; S6. The classified lesion area enters the detection module. Based on the three-dimensional single-stage detection network of the multi-task joint anchor mechanism, the weighted regression fusion method is used to improve the detection accuracy and real-time performance of lesion cells in microscopic hyperspectral images. S7. The test results obtained by the detection module are verified by the clinician based on the pathological results obtained from the pathology expert.

2. The feature extraction and recognition method for skin lesion hyperspectral image classification according to claim 1 is characterized in that: The preprocessing in S1 reduces data redundancy and improves image quality by means of spectral normalization, denoising, and band selection. Band selection is a method based on mutual information to select spectral bands that are highly correlated with pathological tissue characteristics. The calculation formula of mutual information is as follows: Among them, X and Y represent different bands and pathological tissue characteristics respectively, p(x,y) represents the joint probability distribution, and p(x) and p(y) represent the marginal probability distribution.

3. The feature extraction and recognition method for skin lesion hyperspectral image classification according to claim 1 is characterized in that: The three-dimensional convolutional neural network in S2 includes a multi-channel input layer, a convolution layer, a batch normalization layer, an activation function, an adaptive pooling layer and a fully connected layer.

4. The feature extraction and recognition method for skin lesion hyperspectral image classification according to claim 1 is characterized in that: The step of extracting spectral features in S2 is to extract the spectral features of each pixel in multiple bands using a three-dimensional convolution operation. The spectral feature extraction formula is as follows: S(i,j,k)=(k*I)(i,j,k)=∑ m I(i,j,k)*K(i-m,j-n,k-p); Among them, i, j, k are three-dimensional vector indices, m, n, p are convolution kernel sizes, I(i, j, k) is the input three-dimensional vector, K(i, j, k) is the three-dimensional convolution kernel, and S(i, j, k) is the extracted three-dimensional feature.

5. The feature extraction and recognition method for skin lesion hyperspectral image classification according to claim 1 is characterized in that: The feature fusion module in S3 is used to fuse spectral features and spatial features to form a multi-dimensional feature representation. First, the spectral features and spatial features are extracted through a conventional three-dimensional convolutional neural network, and information from different channels and levels is obtained respectively. Through multi-granularity feature fusion, spectral information and spatial position information of different scales are combined to form a high-dimensional feature representation. At the same time, the features are decoupled based on the mutual information entropy method. The formula for feature fusion is as follows: F = Concat(S,C); Among them, F represents the fused feature vector, S and C represent the spectral feature and spatial feature respectively.

6. The feature extraction and recognition method for skin lesion hyperspectral image classification according to claim 1 is characterized in that: The segmentation module in S4 is based on a segmentation model of a three-dimensional fully convolutional neural network. It achieves accurate segmentation of lesion areas in microscopic hyperspectral images through multi-task joint learning. The segmentation model uses an adaptive loss function to dynamically optimize the segmentation accuracy. The formula of the adaptive loss function is as follows: Among them, y i and They represent the true label and the predicted label respectively, and α is the weight coefficient of the loss function.

7. The feature extraction and recognition method for skin lesion hyperspectral image classification according to claim 1 is characterized in that: The classification formula in the classification module in S5 is as follows: f(x)=softmax(W2*ReLU(W1*x+b1)+b2); Among them, x is the input feature vector, W1 and W2 are weight matrices, and b1 and b2 are bias terms.

8. The feature extraction and recognition method for skin lesion hyperspectral image classification according to claim 1 is characterized in that: The detection module in S6 is based on a three-dimensional single-stage detection network with a multi-task joint anchor mechanism, and improves the detection accuracy and real-time performance of diseased cells in microscopic hyperspectral images through a weighted regression fusion method.

Images