Lung medical image depth fuzzy clustering method based on coarse and fine granularity contrast learning

Abstract

The invention provides a lung medical image depth fuzzy clustering method based on coarse and fine granularity contrast learning, and belongs to the technical field of medical image processing. The technical problems that when a traditional method is used for processing the lung medical image, local features are lost, similar categories cannot be accurately distinguished, and processing data boundaries are fuzzy are solved. According to the technical scheme, the method comprises the following steps: S10, obtaining feature representation of a lung medical image, and clustering to generate a preliminary pseudo tag; s20, acquiring an enhanced image of the lung medical image by using a data enhancement technology; s30, obtaining a fuzzy feature representation; s40, optimizing feature representation in combination with cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss; and S50, obtaining a final lung medical image clustering result. The scheme has the beneficial effects that the relevance between global information and local information can be effectively improved, the feature expression ability is enhanced, and the adaptability of the model to complex data is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing and deep learning technology, and in particular to a deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrast learning. Background Art

[0002] Lung cancer is one of the most lethal cancers worldwide, often accompanied by symptoms such as cough, shortness of breath, chest pain, and weight loss, and may not show obvious symptoms in the early stages, causing many patients to miss the best time for treatment. Since different types of lung cancer differ significantly in pathological features and treatment responses, accurately identifying the type, location, and biological behavior of the tumor is crucial for developing an individualized treatment plan. In recent years, the application of deep learning technology has provided new opportunities for lung medical image analysis, especially in the early diagnosis and subtype classification of lung cancer. By performing automated cluster analysis on lung medical images, accurate classification of different types of tumors can be achieved. This can not only improve the accuracy and efficiency of diagnosis, but also assist doctors in comprehensively evaluating the growth pattern of tumors, further promote the development of personalized treatment, and improve the prognosis of lung cancer patients.

[0003] "An Intelligent Classification System Method and System for Lung Cancer Tissue Pathology Images" proposed an intelligent image classification method based on multiple classifiers, but because this method cannot effectively process unlabeled data, its application is limited in unsupervised or weakly supervised learning scenarios. At the same time, because lung medical images usually have the characteristics of high dimensionality, unstructuredness, and noise interference, a more powerful feature representation method is required for subsequent clustering processes. Therefore, a new method is urgently needed to process unlabeled data and efficiently obtain feature representations, which is of great significance to improving the efficiency and accuracy of lung medical image analysis and promoting the application research of medical images in unlabeled data scenarios. Summary of the invention

[0004] The purpose of the present invention is to provide a deep fuzzy clustering method for lung medical images based on coarse-grained and fine-grained contrast learning, obtain the feature representation of lung medical images and cluster them to generate preliminary pseudo-labels, which is beneficial to improving the quality of pseudo-labels, and use self-attention fuzzy online network and self-attention fuzzy target network to obtain fuzzy feature representation respectively, which is beneficial to enhancing detail feature extraction, and combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss to optimize feature representation, which is beneficial to improving the distinguishability of feature representation. The present invention will provide important medical imaging basis for early diagnosis and subtype classification of lung cancer.

[0005] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is specifically: a deep fuzzy clustering method of lung medical images based on coarse-fine granularity contrast learning, comprising the following steps:

[0006] S10, obtaining feature representation of lung medical images and clustering to generate preliminary pseudo labels;

[0007] S20, data enhancement to obtain an enhanced image of the lung medical image;

[0008] S30, using the self-attention fuzzy online network and the self-attention fuzzy target network to obtain fuzzy feature representations respectively;

[0009] S40, combining cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss to optimize feature representation;

[0010] S50. Based on the preliminary clustering results of the current network and the optimized feature representation, the clustering process is repeated, the clustering results and pseudo-labels are adjusted, and the pseudo-labels are used to guide the subsequent self-attention fuzzy feature representation learning; the clustering results and target network parameters are iteratively updated to obtain the final lung medical image clustering results.

[0011] As the deep fuzzy clustering method of lung medical images based on coarse-fine granularity contrast learning provided by the present invention, step S10 includes the following steps:

[0012] S11. Input a batch of lung medical images X∈R N×H×W , where N is the number of samples in the current batch, H and W are the height and width of the lung medical image, respectively, and X contains the visual features of all images in the batch. The feature matrix is ​​obtained using the self-attention fuzzy target network. d1 is the feature dimension after the self-attention blurred target (online) network, each row represents the feature of a sample;

[0013] S12, the obtained feature matrix Z is further updated through the target projector network g2(·) to obtain the updated feature matrix d2 is the feature dimension after passing through the projector network g2(·);

[0014] S13, obtain the feature matrix of all samples in the current data set through S12, perform fuzzy c-means clustering on the feature matrix of all samples, and use the clustering result of each sample as a pseudo label To guide the subsequent self-attention fuzzy feature representation learning, where N all is the number of all samples in the lung medical image dataset.

[0015] As the lung medical image deep fuzzy clustering method based on coarse-fine granularity contrast learning provided by the present invention, step S20 includes the following steps:

[0016] S21. Set the enhancement method of lung medical images. The deep fuzzy clustering method of lung medical images based on coarse-grained contrast learning adopts a variety of data enhancement methods, including random cropping, flipping, color enhancement and Gaussian blur, as follows:

[0017] Random cropping is to randomly crop an image and resize it to a specified size. First, a random scale scale∈(s min ,s max ), and then the area is cropped, where s min and max The minimum and maximum ratios of the cropped area to the original image area are calculated, and finally the image is scaled to the specified target size.

[0018] Flipping is to flip the image horizontally with a probability of 50%, and the pixel position of the image changes from (x, y) to (W p -x,y), where (x,y) is the coordinate position, W p is the width of the image;

[0019] Color enhancement is to randomly apply a color enhancement operation with 80% probability, which is achieved by adjusting the brightness, contrast, saturation and hue of the image;

[0020] Gaussian blur is applied with a probability of 50%. The convolution kernel used by Gaussian blur is a two-dimensional Gaussian distribution, expressed as:

[0021]

[0022] Among them, G(a,b,σ b ) is the two-dimensional Gaussian distribution of the Gaussian blur convolution kernel, a, b are the offsets of each pixel in the image relative to the center, σ b is the standard deviation of the Gaussian kernel, controlling the degree of blur;

[0023] S22. The enhanced image of the lung medical image is converted into a tensor, and is converted into a format suitable for deep learning model processing through standardization. The standardization formula is expressed as follows:

[0024]

[0025] Among them, x p ' is the pixel value of the standardized image channel, x p is the value of each pixel in the image channel, μ n is the mean of the image channels, σ n is the standard deviation of the image channel;

[0026] S23, using the above random enhancement settings, perform two random enhancements and input the original version of the current batch of lung medical images X∈RN×H×W , and obtain two sets of two different enhanced versions of lung medical images X 1 ∈R N×H×W and X 2 ∈R N×H×W .

[0027] As the lung medical image deep fuzzy clustering method based on coarse-fine granularity contrast learning provided by the present invention, step S30 includes the following steps:

[0028] S31. Use the self-attention fuzzy online network and the self-attention fuzzy target network to extract two different enhanced versions of lung medical image X 1 and X 2 Visual features of the and Where N is the number of samples in the current batch, and d1 is the feature dimension after the self-attention blurred target (online) network:

[0029] Z 1 '=encoder online (X 1 )(3)

[0030] Z 2 '=encoder target (X 2 )(4)

[0031] Among them, X 1 and X 2 They are enhanced images, encoder online (·) is the self-attention fuzzy online encoder network, encoder target (·) is the self-attention fuzzy target encoder network, Z 1 ' and Z 2 ' is the acquired feature matrix;

[0032] S32. For the feature matrix The self-attention mechanism is used to weight and adjust the feature representation of each sample by calculating the relationship between each sample in the data. Input feature matrix Z 1 ' is mapped to query, key and value spaces. The mapping process is expressed as:

[0033] Q=Z 1 'W Q (5)

[0034] K=Z 1 'W K (6)

[0035] V=Z 1 'W V (7)

[0036] Among them, Z 1 ' is the input feature matrix, is the query matrix, is the key matrix, is the value matrix, W Q , W K and W V They are the weight matrices learned by mapping to query, key, and value spaces respectively;

[0037] S33. Calculate the similarity between the query matrix and the key matrix through the dot product, and get the attention weight expressed as:

[0038]

[0039] Among them, A∈R N×N is the similarity matrix, d1 is the feature dimension after the self-attention fuzzy target (online) network; the similarity matrix A is normalized by the softmax function, and the attention score is expressed as A'∈R N×N :

[0040] A'=softmax(A)(9)

[0041] Among them, softmax(·) is an activation function; the value matrix V is weighted by the attention score A', and the process is expressed as:

[0042]

[0043] in, is the weighted feature matrix;

[0044] S34, the obtained weighted feature matrix Fuzzy processing is performed, and M membership functions are applied to each feature point. The membership of each feature is calculated by the membership function. The membership function used in this method is a Gaussian function, which is defined as:

[0045]

[0046] in, is the eigenvalue of the weighted feature of the i-th sample in the feature space of the first enhanced version in the j-th dimension, μ k and σ k are the mean and standard deviation of the kth Gaussian membership function, μ k and σ k It is a trainable parameter, and the membership function parameter μ of all feature points of the same sample k and σ k are the same;

[0047] S35, applying fuzzy rules to aggregate the membership matrix, and obtaining the fuzzy eigenvalue of each feature point:

[0048]

[0049] in, is the fuzzy eigenvalue of the weighted feature of the i-th sample in the j-th dimension, and “Π” is the multiplication symbol; the fuzzy eigenvalue is processed by batch normalization to constrain its dynamic range, and then the addition operation is used to integrate the fuzzy eigenvalue with the original feature Z' to obtain the final fuzzy feature matrix The obtained feature matrix is ​​passed through the online projector network g1(·) to obtain the updated feature matrix

[0050] S36, Lung Medical Image Enhancement ImageX 1 After the self-attention fuzzy online network, the fuzzy feature matrix Z is obtained 1 , through the online projector network g1(·), the updated fuzzy feature matrix g1(Z 1 ), the process is as described in S31-35, and the fuzzy feature matrix Z 1 The acquisition process is the same as that of the lung medical image enhancement image X 2 After the self-attention fuzzy target network, the fuzzy feature matrix is ​​obtained Through the target projector network g2(·), the updated fuzzy feature matrix is ​​obtained

[0051] As the lung medical image deep fuzzy clustering method based on coarse-fine granularity contrast learning provided by the present invention, step S40 includes the following steps:

[0052] S41, for the feature matrix g1(Z 1 ) and g2(Z 2 ), obtain K prototypes from an enhanced version of the feature space Another enhanced version of the feature space also obtained K prototypes The prototype calculation formula for each cluster in the feature space of the two enhanced versions of each batch B is:

[0053]

[0054] Where v is 1 or 2, representing the feature space of two enhanced versions, m = 1, 2, ..., K, representing K prototypes, is the feature vector of the mth prototype in the feature space of the vth enhanced version, is the feature vector of the i-th sample in the feature space of the v-th enhanced version, is the fuzzy feature representation of the i-th sample in the feature space of the v-th enhanced version, u im is the probability that the i-th sample belongs to the m-th cluster, ||·||2 is the L2 norm, which is used to calculate the length of the vector;

[0055] S42. Use the prototype obtained by the above calculation to calculate the cluster-level coarse-grained contrast loss, and the formula is expressed as:

[0056]

[0057]

[0058]

[0059] in, and are the feature vectors of the mth prototype in the feature space of the first and second enhanced versions, respectively, and are the feature vectors of the jth prototype in the feature space of the first and second enhanced versions, respectively. “·” represents the vector dot multiplication operation, τ c is the temperature coefficient, which controls the degree of scaling in the similarity calculation, and are the cluster-level coarse-grained contrastive losses calculated in the feature space of the first and second enhanced versions, L c is the overall cluster-level coarse-grained contrast loss;

[0060] S43. Calculate the sample-level fine-grained contrast loss, and the formula is expressed as:

[0061]

[0062]

[0063]

[0064] in, and are the feature vectors of the i-th sample in the feature space of the first and second enhanced versions, respectively. is relative to For all positive samples in the feature space of the second enhanced version, is relative to For all positive samples in the feature space of the second enhanced version, z n is all samples except the i-th sample, τ sis the temperature coefficient, A(i) is the set of all samples in the feature space of the two enhanced versions, P(i) is all positive samples corresponding to the current sample, |P(i)| is the number of all positive samples corresponding to the current sample, N is the number of samples in the current batch, and are the sample-level coarse-grained contrastive losses calculated in the feature space of the first and second enhanced versions, L s is the overall cluster-level coarse-grained contrast loss;

[0065] S44, the feature matrix g1(Z 1 ) passes through the predictor network f(·) to obtain the prediction feature matrix f(g1(Z 1 )), the self-attention fuzzy online network is to predict the self-attention fuzzy target network's representation of the same image but different enhanced versions, and the mean square error loss L between the prediction of the self-attention fuzzy online network and the feature matrix mapped by the target network mse It is expressed as:

[0066]

[0067] where f(·) is the predictor network, is the square of the L2 norm;

[0068] S45. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss to optimize the feature representation described in step S30, update the self-attention fuzzy online network parameters through back propagation, and update the self-attention fuzzy target network through momentum.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] (1) The present invention generates preliminary pseudo-labels by performing feature representation and clustering on lung medical images. This process can guide subsequent feature representation learning, avoid dependence on a large amount of labeled data, and significantly improve the efficiency and effect of unsupervised learning. Pseudo-labels can not only guide subsequent feature optimization, but also provide a reliable initial framework for further data enhancement and fuzzy feature representation.

[0071] (2) The present invention introduces a self-attention fuzzy feature representation module, which represents image features through a self-attention fuzzy online network and a self-attention fuzzy target network. This innovative method enables the model to better capture the relationship between image details and global information when processing lung medical images. By optimizing the fuzzy features, the model can more accurately capture subtle changes in key areas of the image, improving the accuracy of image clustering.

[0072] (3) The invention optimizes the training process of feature representation by combining cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss. Coarse-grained contrast loss optimizes the overall relationship between image clusters, while fine-grained contrast loss focuses on the detailed features inside the image, thereby improving the learning ability of the model at different levels. This further improves the reliability and accuracy of medical image analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0074] Figure 1 This is a patent flowchart of the deep fuzzy clustering method of lung medical images based on coarse and fine granularity contrast learning of the present invention.

[0075] Figure 2 This is the patent framework diagram of the deep fuzzy clustering method of lung medical images based on coarse and fine granularity contrast learning of the present invention.

[0076] Figure 3 This is a self-attention fuzzy feature representation framework diagram of the deep fuzzy clustering method of lung medical images based on coarse-fine granularity contrast learning of the present invention.

[0077] Figure 4 This is a partial result diagram of Example 1 of the deep fuzzy clustering method for lung medical images based on coarse- and fine-grained contrast learning of the present invention.

[0078] Figure 5 This is a partial result diagram of Example 2 of the deep fuzzy clustering method for lung medical images based on coarse- and fine-grained contrast learning of the present invention. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Of course, the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.

[0080] Example 1

[0081] See also Figure 1-5 As shown, the technical solution provided by this embodiment is a deep fuzzy clustering method of lung medical images based on coarse-fine granularity contrast learning, comprising the following steps:

[0082] S10, obtaining feature representation of lung medical images and clustering to generate preliminary pseudo labels;

[0083] S20, obtaining an enhanced image of the lung medical image using data enhancement technology;

[0084] S30, using the self-attention fuzzy online network and the self-attention fuzzy target network to obtain fuzzy feature representations respectively;

[0085] S40, combining cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss to optimize feature representation;

[0086] S50, iteratively update the clustering results and the self-attention fuzzy target network parameters to obtain the final lung medical image clustering results.

[0087] Step S10 includes the following steps:

[0088] S11. Take the lung CT scan image dataset from Kaggle as an example. The dataset contains four types of lung medical images, including pancreatic cancer, large cell carcinoma, squamous cell carcinoma, and normal tissue, with a total of 1,000 images. Input a batch of lung medical images X∈R 128×224×224 , where a batch contains 128 lung medical images, the height and width of the lung medical images are 224 and 224 respectively, X contains the visual features of all images in the batch, and the feature matrix Z∈R is obtained using the self-attention fuzzy target network 128×512 , the feature dimension after the self-attention fuzzy target (online) network is 512, each row represents the feature of a sample, X is shown in formula (1):

[0089]

[0090] S12, the obtained feature matrix Z is further updated through the target projector network g2(·), and the updated feature matrix g2(Z)∈R is obtained. 128×256 , d2 is the feature dimension after passing through the projector network g2(·), g2(Z) is as shown in formula (2):

[0091]

[0092] S13, obtain the feature matrix of all samples in the current data set through S12, perform fuzzy c-means clustering on the feature matrix of all samples, and use the clustering result of each sample as a pseudo label L pseudo ∈R 1000×1 To guide the subsequent self-attention fuzzy feature representation learning, where 1000 is the number of all samples in the lung medical image dataset, and L pseudo As shown in formula (3):

[0093]

[0094] Step S20 includes the following steps:

[0095] S21. Set the enhancement method of lung medical images. The deep fuzzy clustering method of lung medical images based on coarse-grained and fine-grained contrast learning adopts a variety of data enhancement methods, including random cropping, flipping, color enhancement and Gaussian blur.

[0096] S22. The enhanced image of the lung medical image is converted into a tensor and transformed into a format suitable for processing by a deep learning model through standardization.

[0097] S23, using the above random enhancement settings, perform two random enhancements and input the original version of the current batch of lung medical images X∈R 128×224×224 , and obtain two sets of different enhanced versions of lung medical images X 1 ∈R 128×224×224 and X 2 ∈R 128 ×224×224 ,X 1 and X 2 As shown in formulas (4) and (5):

[0098]

[0099]

[0100] Step S30 includes the following steps:

[0101] S31. Use the self-attention fuzzy online network and the self-attention fuzzy target network to extract two different enhanced versions of lung medical image X 1 and X 2 Visual features of the and Where N is the number of samples in the current batch, and d1 is the feature dimension after the self-attention blurred target (online) network:

[0102] Z 1 '=encoder online (X 1 )(6)

[0103] Z 2 '=encoder target (X 2 )(7)

[0104]

[0105]

[0106] Among them, X 1 and X 2 They are enhanced images, encoder online(·) is the self-attention fuzzy online encoder network, encoder target (·) is the self-attention fuzzy target encoder network, Z 1 ' and Z 2 ' is the acquired feature matrix;

[0107] S32. For the feature matrix The self-attention mechanism is used to weight and adjust the feature representation of each sample by calculating the relationship between each sample in the data. Input feature matrix Z 1 ' is mapped to query, key and value spaces. The mapping process is expressed as:

[0108] Q=Z 1 'W Q (10)

[0109]

[0110] K=Z 1 'W K (12)

[0111]

[0112] V=Z 1 'W V (14)

[0113]

[0114] Among them, Z 1 ' is the input feature matrix, is the query matrix, is the key matrix, is the value matrix, W Q , W K and W V They are the weight matrices learned by mapping to query, key, and value spaces respectively;

[0115] S33. Calculate the similarity between the query matrix and the key matrix through the dot product, and get the attention weight expressed as:

[0116]

[0117]

[0118] Among them, A∈R N×N is the similarity matrix, d1 is the feature dimension after the self-attention fuzzy target (online) network; the similarity matrix A is normalized by the softmax function, and the attention score is expressed as A'∈R N×N :

[0119] A'=softmax(A)(18)

[0120]

[0121] Among them, softmax(·) is an activation function; the value matrix V is weighted by the attention score A', and the process is expressed as:

[0122]

[0123]

[0124] in, is the weighted feature matrix;

[0125] S34, the obtained weighted feature matrix Fuzzy processing is performed, and M membership functions are applied to each feature point. The membership of each feature is calculated by the membership function. The membership function used in this method is a Gaussian function, which is defined as:

[0126]

[0127]

[0128] in, is the eigenvalue of the weighted feature of the i-th sample in the feature space of the first enhanced version in the j-th dimension, μ k and σ k are the mean and standard deviation of the kth Gaussian membership function, μ k and σ k It is a trainable parameter, and the membership function parameter μ of all feature points of the same sample k and σ k are the same;

[0129] S35, applying fuzzy rules to aggregate the membership matrix, and obtaining the fuzzy eigenvalue of each feature point:

[0130]

[0131]

[0132] in, is the fuzzy eigenvalue of the weighted feature of the i-th sample in the j-th dimension, and “Π” is the multiplication symbol; the fuzzy eigenvalue is processed by batch normalization to constrain its dynamic range, and then the addition operation is used to integrate the fuzzy eigenvalue with the original feature Z' to obtain the final fuzzy feature matrix The obtained feature matrix is ​​passed through the online projector network g1(·) to obtain the updated feature matrix

[0133] S36, Lung Medical Image Enhancement ImageX 1 After the self-attention fuzzy online network, the fuzzy feature matrix Z is obtained 1 , through the online projector network g1(·), the updated fuzzy feature matrix g1(Z 1 ), the process is as described in S31-35, and the fuzzy feature matrix Z 1 The acquisition process is the same as that of the lung medical image enhancement image X 2 After the self-attention fuzzy target network, the fuzzy feature matrix is ​​obtained Through the target projector network g2(·), the updated fuzzy feature matrix is ​​obtained

[0134] Step S40 includes the following steps:

[0135] S41, for the feature matrix g1(Z 1 ) and g2(Z 2 ), 4 prototypes are obtained from an enhanced version of the feature space Another enhanced version of the feature space also obtained 4 prototypes The prototype calculation formula for each cluster in the feature space of the two enhanced versions of each batch B is:

[0136]

[0137]

[0138]

[0139] Where v is 1 or 2, representing the feature space of two enhanced versions, m = 1, 2, ..., K, representing K prototypes, is the feature vector of the mth prototype in the feature space of the vth enhanced version, is the feature vector of the i-th sample in the feature space of the v-th enhanced version, is the fuzzy feature representation of the i-th sample in the feature space of the v-th enhanced version, u im is the probability that the i-th sample belongs to the m-th cluster, ||·||2 is the L2 norm, which is used to calculate the length of the vector;

[0140] S42. Use the prototype obtained by the above calculation to calculate the cluster-level coarse-grained contrast loss, and the formula is expressed as:

[0141]

[0142]

[0143]

[0144] in, and are the feature vectors of the mth prototype in the feature space of the first and second enhanced versions, respectively, and are the feature vectors of the jth prototype in the feature space of the first and second enhanced versions, respectively. “·” represents the vector dot multiplication operation, τ c is the temperature coefficient, which controls the degree of scaling in the similarity calculation, and are the cluster-level coarse-grained contrastive losses calculated in the feature space of the first and second enhanced versions, L c is the overall cluster-level coarse-grained contrast loss;

[0145] S43. Calculate the sample-level fine-grained contrast loss, and the formula is expressed as:

[0146]

[0147]

[0148] in, and are the feature vectors of the i-th sample in the feature space of the first and second enhanced versions, respectively. is relative to For all positive samples in the feature space of the second enhanced version, is relative to For all positive samples in the feature space of the second enhanced version, z n is all samples except the i-th sample, τ s is the temperature coefficient, A(i) is the set of all samples in the feature space of the two enhanced versions, P(i) is all positive samples corresponding to the current sample, |P(i)| is the number of all positive samples corresponding to the current sample, N is the number of samples in the current batch, and are the sample-level coarse-grained contrastive losses calculated in the feature space of the first and second enhanced versions, L s is the overall cluster-level coarse-grained contrast loss;

[0149] S44, the feature matrix g1(Z 1 ) passes through the predictor network f(·) to obtain the prediction feature matrix f(g1(Z 1)), the self-attention fuzzy online network is to predict the self-attention fuzzy target network's representation of the same image but different enhanced versions, and the mean square error loss L between the prediction of the self-attention fuzzy online network and the feature matrix mapped by the target network mse It is expressed as:

[0150]

[0151] where f(·) is the predictor network, is the square of the L2 norm;

[0152] S45. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss to optimize the feature representation described in step S30, update the self-attention fuzzy online network parameters through back propagation, and update the self-attention fuzzy target network through momentum.

[0153] Step S50 includes the following steps:

[0154] S51. Based on the preliminary clustering results of the current network and the optimized feature representation, the clustering process is repeated, the clustering results and pseudo-labels are adjusted, and the pseudo-labels are used to guide the subsequent self-attention fuzzy feature representation learning; the clustering results and target network parameters are iteratively updated to obtain the final lung medical image clustering results.

[0155] The lung CT scan image dataset is input into three different models for clustering tasks: the first two are traditional clustering models, and the third is our proposed method. Calculate the clustering accuracy of all samples True indicates the number of samples correctly clustered by the model, and False indicates the number of samples correctly clustered by the model. The results show that the clustering results of this method are better.

[0156] Table 1

[0157]

[0158] Example 2

[0159] See also Figure 1-5 As shown, the technical solution provided by this embodiment is a deep fuzzy clustering method of lung medical images based on coarse-fine granularity contrast learning, comprising the following steps:

[0160] S10, obtaining feature representation of lung medical images and clustering to generate preliminary pseudo labels;

[0161] S20, obtaining an enhanced image of the lung medical image using data enhancement technology;

[0162] S30, using the self-attention fuzzy online network and the self-attention fuzzy target network to obtain fuzzy feature representations respectively;

[0163] S40, combining cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss to optimize feature representation;

[0164] S50, iteratively update the clustering results and the self-attention fuzzy target network parameters to obtain the final lung medical image clustering results.

[0165] Step S10 includes the following steps:

[0166] S11. Take the X-ray image pneumonia dataset from Kaggle as an example. This dataset contains three types of lung medical images, including normal, viral pneumonia, and bacterial pneumonia, with a total of 5284 images. Input a batch of lung medical images X∈R 128×224×224 , where a batch contains 128 lung medical images, the height and width of the lung medical images are 224 and 224 respectively, X contains the visual features of all images in the batch, and the feature matrix Z∈R is obtained using the self-attention fuzzy target network 128×512 , the feature dimension after the self-attention fuzzy target (online) network is 512, each row represents the feature of a sample, X is shown in formula (1):

[0167]

[0168] S12, the obtained feature matrix Z is further updated through the target projector network g2(·), and the updated feature matrix g2(Z)∈R is obtained. 128×256 , d2 is the feature dimension after passing through the projector network g2(·), g2(Z) is as shown in formula (2):

[0169]

[0170] S13, obtain the feature matrix of all samples in the current data set through S12, perform fuzzy c-means clustering on the feature matrix of all samples, and use the clustering result of each sample as a pseudo label L pseudo ∈R 1000×1 To guide the subsequent self-attention fuzzy feature representation learning, where 5284 is the number of all samples in the lung medical image dataset, L pseudo As shown in formula (3):

[0171]

[0172] Step S20 includes the following steps:

[0173] S21. Set the enhancement method of lung medical images. The deep fuzzy clustering method of lung medical images based on coarse-grained and fine-grained contrast learning adopts a variety of data enhancement methods, including random cropping, flipping, color enhancement and Gaussian blur.

[0174] S22. The enhanced image of the lung medical image is converted into a tensor and transformed into a format suitable for processing by a deep learning model through standardization.

[0175] S23, using the above random enhancement settings, perform two random enhancements and input the original version of the current batch of lung medical images X∈R 128×224×224 , and obtain two sets of different enhanced versions of lung medical images X 1 ∈R 128×224×224 and X 2 ∈R 128 ×224×224 ,X 1 and X 2 As shown in formulas (4) and (5):

[0176]

[0177]

[0178] Step S30 includes the following steps:

[0179] S31. Use the self-attention fuzzy online network and the self-attention fuzzy target network to extract two different enhanced versions of lung medical image X 1 and X 2 Visual features of the and Where N is the number of samples in the current batch, and d1 is the feature dimension after the self-attention blurred target (online) network:

[0180]

[0181]

[0182] S32. For the feature matrix The self-attention mechanism is used to weight and adjust the feature representation of each sample by calculating the relationship between each sample in the data. Input feature matrix Z 1 ' is mapped to query, key and value spaces. The mapping process is expressed as:

[0183]

[0184]

[0185]

[0186] S33. Calculate the similarity between the query matrix and the key matrix through the dot product, and get the attention weight expressed as:

[0187]

[0188] The similarity matrix A is normalized by the softmax function, and the attention score is expressed as A'∈R N×N :

[0189]

[0190] The value matrix V is weighted using the attention score A', and the process is expressed as:

[0191]

[0192] S34, the obtained weighted feature matrix Fuzzy processing is performed, and M membership functions are applied to each feature point. The membership of each feature is calculated by the membership function. The membership function used by this method is a Gaussian function:

[0193]

[0194] S35, applying fuzzy rules to aggregate the membership matrix, and obtaining the fuzzy eigenvalue of each feature point:

[0195]

[0196] The fuzzy eigenvalues ​​are processed by batch normalization to constrain their dynamic range. Then, the fuzzy eigenvalues ​​are integrated with the original features Z' by addition operation to obtain the final fuzzy feature matrix The obtained feature matrix is ​​passed through the online projector network g1(·) to obtain the updated feature matrix

[0197] S36, Lung Medical Image Enhancement ImageX 1 After the self-attention fuzzy online network, the fuzzy feature matrix Z is obtained 1 , through the online projector network g1(·), the updated fuzzy feature matrix g1(Z 1 ), the process is as described in S31-35, and the fuzzy feature matrix Z 1 The acquisition process is the same as that of the lung medical image enhancement image X 2 After the self-attention fuzzy target network, the fuzzy feature matrix is ​​obtained Through the target projector network g2(·), the updated fuzzy feature matrix is ​​obtained

[0198] Step S40 includes the following steps:

[0199] S41, for the feature matrix g1(Z 1 ) and g2(Z 2 ), and obtain 3 prototypes from an enhanced version of the feature space Another enhanced version of the feature space also obtained 3 prototypes The prototype calculation formula for each cluster in the feature space of the two enhanced versions of each batch B is:

[0200]

[0201]

[0202] S42. Use the prototype obtained by the above calculation to calculate the cluster-level coarse-grained contrast loss.

[0203] S43. Calculate sample-level fine-grained contrast loss.

[0204] S44. Calculate the mean square error loss of the feature matrix of the self-attention blurred online network's prediction and the target network mapping.

[0205] S45. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss to optimize the feature representation described in step S30, update the self-attention fuzzy online network parameters through back propagation, and update the self-attention fuzzy target network through momentum.

[0206] Step S50 includes the following steps:

[0207] S51, iteratively update the clustering results and target network parameters to obtain the final lung medical image clustering results.

[0208] The X-ray image pneumonia dataset is fed into three different models for clustering tasks: the first two are traditional clustering models, and the third is our proposed method. The clustering accuracy of all samples is calculated.

[0209] Table 2

[0210]

[0211] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A deep fuzzy clustering method for lung medical images based on coarse-grained and fine-grained contrast learning, characterized in that: The following steps are involved: S10, obtaining feature representation of lung medical images and clustering to generate preliminary pseudo labels; S20, obtaining an enhanced image of the lung medical image using data enhancement technology; S30, using the self-attention fuzzy online network and the self-attention fuzzy target network to obtain fuzzy feature representations respectively; S40, combining cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss to optimize feature representation; S50, iteratively update the clustering results and the self-attention fuzzy target network parameters to obtain the final lung medical image clustering results.

2. The deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrast learning according to claim 1 is characterized in that: The step S10 comprises the following steps: S11. Input a batch of lung medical images X∈R N×H×W , where N is the number of samples in the current batch, H and W are the height and width of the lung medical image, respectively, and X contains the visual features of all images in the batch. The feature matrix is ​​obtained using the self-attention fuzzy target network. d1 is the feature dimension after the target online network is blurred by self-attention, and each row represents the feature of a sample; S12, the obtained feature matrix Z is further updated through the target projector network g2(·) to obtain the updated feature matrix d2 is the feature dimension after passing through the projector network g2(·); S13: Obtain the feature matrix of all samples in the current data set through step S12, perform fuzzy c-means clustering on the feature matrix of all samples, and use the clustering result of each sample as a pseudo label. To guide the subsequent self-attention fuzzy feature representation learning, where N all is the number of all samples in the lung medical image dataset.

3. The deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrast learning according to claim 1 is characterized in that: The step S20 comprises the following steps: S21. Set the enhancement method of lung medical images. The deep fuzzy clustering method of lung medical images based on coarse-grained contrast learning adopts a variety of data enhancement methods, including random cropping, flipping, color enhancement and Gaussian blur, as follows: Random cropping is to randomly crop an image and resize it to a specified size. First, a random scale scale∈(s min ,s max ), and then the area is cropped, where s min and max The minimum and maximum ratios of the cropped area to the original image area are calculated, and finally the image is scaled to the specified target size. Flipping is to flip the image horizontally with a probability of 50%, and the pixel position of the image changes from (x, y) to (W p -x,y), where (x,y) is the coordinate position, W p is the width of the image; Color enhancement is to randomly apply a color enhancement operation with 80% probability, which is achieved by adjusting the brightness, contrast, saturation and hue of the image; Gaussian blur is applied with a probability of 50%. The convolution kernel used by Gaussian blur is a two-dimensional Gaussian distribution, expressed as: Among them, G(a,b,σ b ) is the two-dimensional Gaussian distribution of the Gaussian blur convolution kernel, a, b are the offsets of each pixel in the image relative to the center, σ b is the standard deviation of the Gaussian kernel, controlling the degree of blur; S22. The enhanced image of the lung medical image is converted into a tensor, and is converted into a format suitable for deep learning model processing through standardization. The standardization formula is expressed as follows: Among them, x p ' is the pixel value of the standardized image channel, x p is the value of each pixel in the image channel, μ n is the mean of the image channels, σ n is the standard deviation of the image channel; S23, using the above random enhancement settings, perform two random enhancements and input the original version of the current batch of lung medical images X∈R N×H×W , and obtain two sets of two different enhanced versions of lung medical images X 1 ∈R N×H×W and X 2 ∈R N×H×W .

4. The deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrast learning according to claim 1 is characterized in that: The step S30 includes the following steps: S31. Use the self-attention fuzzy online network and the self-attention fuzzy target network to extract two different enhanced versions of lung medical image X 1 and X 2 Visual features of the and Where N is the number of samples in the current batch, and d1 is the feature dimension after the self-attention blurred target online network: Z 1 '=encoder online (X 1 )(3) Z 2 '=encoder target (X 2 )(4) Among them, X 1 and X 2 They are enhanced images, encoder online (·) is the self-attention fuzzy online encoder network, encoder target (·) is the self-attention fuzzy target encoder network, Z 1 ' and Z 2 ' is the acquired feature matrix; S32. For the feature matrix The self-attention mechanism is used to weight and adjust the feature representation method of each sample by calculating the relationship between each sample in the data, and the input feature matrix Z 1 ' is mapped to query, key and value spaces. The mapping process is expressed as: Q=Z 1 'W Q (5) K=Z 1 'IN K (6) V=Z 1 'W V (7) Among them, Z 1 ' is the input feature matrix, is the query matrix, is the key matrix, is the value matrix, W Q , W K and W V They are the weight matrices learned by mapping to query, key, and value spaces respectively; S33. Calculate the similarity between the query matrix and the key matrix through the dot product, and get the attention weight expressed as: Among them, A∈R N×N is the similarity matrix, d1 is the feature dimension after the target online network is blurred by self-attention; the similarity matrix A is normalized by the softmax function, and the attention score is expressed as A'∈R N×N : A'=softmax(A) (9) Among them, softmax(·) is an activation function; the value matrix V is weighted by the attention score A', and the process is expressed as: in, is the weighted feature matrix; S34, the obtained weighted feature matrix i=1,2,…,N,j=1,2,…,d1,fuzzy processing is performed, M membership functions are applied to each feature point, and the membership of each feature is calculated by the membership function. The membership function used in this method is a Gaussian function, which is defined as: Where i = 1, 2, ..., N, j = 1, 2, ..., d1, k = 1, 2, ..., M, is the eigenvalue of the weighted feature of the i-th sample in the feature space of the first enhanced version in the j-th dimension, μ k and σ k are the mean and standard deviation of the kth Gaussian membership function, μ k and σ k It is a trainable parameter, and the membership function parameter μ of all feature points of the same sample k and σ k are the same; S35, applying fuzzy rules to aggregate the membership matrix, and obtaining the fuzzy eigenvalue of each feature point: in, is the fuzzy eigenvalue of the weighted feature of the i-th sample in the j-th dimension, "Π" is the multiplication symbol; the fuzzy eigenvalue is processed by batch normalization to constrain its dynamic range, and then the addition operation is used to integrate the fuzzy eigenvalue with the original feature Z' to obtain the final fuzzy feature matrix The obtained feature matrix is ​​passed through the online projector network g1(·) to obtain the updated feature matrix S36, Lung Medical Image Enhancement ImageX 1 After the self-attention fuzzy online network, the fuzzy feature matrix Z is obtained 1 , through the online projector network g1(·), the updated fuzzy feature matrix g1(Z 1 ), the process is as described in step S31 to step 35, and the fuzzy feature matrix Z 1 The acquisition process is the same as that of the lung medical image enhancement image X 2 After the self-attention fuzzy target network, the fuzzy feature matrix is ​​obtained Through the target projector network g2(·), the updated fuzzy feature matrix is ​​obtained 5. The deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrast learning according to claim 1, characterized in that: The step S40 comprises the following steps: S41, for the feature matrix g1(Z 1 ) and g2(Z 2 ), obtain K prototypes from an enhanced version of the feature space Another enhanced version of the feature space also obtained K prototypes The prototype calculation formula for each cluster in the feature space of the two enhanced versions of each batch B is: Where v is 1 or 2, representing the feature space of two enhanced versions, m = 1, 2, ..., K, representing K prototypes, is the feature vector of the mth prototype in the feature space of the vth enhanced version, is the feature vector of the i-th sample in the feature space of the v-th enhanced version, is the fuzzy feature representation of the i-th sample in the feature space of the v-th enhanced version, u im is the probability that the i-th sample belongs to the m-th cluster, ||·||2 is the L2 norm, which is used to calculate the length of the vector; S42. Use the prototype obtained by the above calculation to calculate the cluster-level coarse-grained contrast loss, and the formula is expressed as: in, and are the feature vectors of the mth prototype in the feature space of the first and second enhanced versions, respectively, and are the feature vectors of the jth prototype in the feature space of the first and second enhanced versions, respectively. “·” represents the vector dot multiplication operation, τ c is the temperature coefficient, which controls the degree of scaling in the similarity calculation, and are the cluster-level coarse-grained contrastive losses calculated in the feature space of the first and second enhanced versions, L c is the overall cluster-level coarse-grained contrast loss; S43. Calculate the sample-level fine-grained contrast loss, and the formula is expressed as: in, and are the feature vectors of the i-th sample in the feature space of the first and second enhanced versions, respectively. is relative to For all positive samples in the feature space of the second enhanced version, is relative to For all positive samples in the feature space of the second enhanced version, z n is all samples except the i-th sample, τ s is the temperature coefficient, A(i) is the set of all samples in the feature space of the two enhanced versions, P(i) is all positive samples corresponding to the current sample, |P(i)| is the number of all positive samples corresponding to the current sample, N is the number of samples in the current batch, and are the sample-level coarse-grained contrastive losses calculated in the feature space of the first and second enhanced versions, L s is the overall cluster-level coarse-grained contrast loss; S44, the feature matrix g1(Z 1 ) passes through the predictor network f(·) to obtain the prediction feature matrix f(g1(Z 1 )), the self-attention fuzzy online network is to predict the self-attention fuzzy target network's representation of the same image but different enhanced versions, and the mean square error loss L between the prediction of the self-attention fuzzy online network and the feature matrix mapped by the target network mse It is expressed as: where f(·) is the predictor network, is the square of the L2 norm; S45. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss to optimize the feature representation described in step S30, update the self-attention fuzzy online network parameters through back propagation, and update the self-attention fuzzy target network through momentum.

6. According to the deep fuzzy clustering method of lung medical images based on coarse-fine granularity contrast learning according to claim 1, in the step S50, based on the preliminary clustering results of the current network and the optimized feature representation, the clustering process is re-performed, the clustering results and pseudo labels are adjusted, and the pseudo labels are used to guide the subsequent self-attention fuzzy feature representation learning; the clustering results and target network parameters are iteratively updated to obtain the final lung medical image clustering results.

Images