A Deep Fuzzy Clustering Method for Lung Medical Images Based on Coarse-to-Fine Granularity Contrast Learning

Abstract

The application provides a lung medical image deep blur clustering method based on coarse and fine granularity contrast learning, and belongs to the technical field of medical image processing. The technical problems of local feature loss and inability to accurately distinguish similar categories and process data boundary blur when a traditional method is used to process lung medical images are solved. The technical scheme comprises the following steps: S10, obtaining feature representation of a lung medical image and clustering to generate preliminary pseudo labels; S20, obtaining enhanced images of the lung medical image by using a data enhancement technique; S30, obtaining blur feature representation; S40, combining cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss and mean square error loss to optimize the feature representation; and S50, obtaining a final lung medical image clustering result. The beneficial effects of the scheme are that the correlation between global and local information can be effectively improved, the expression ability of features can be enhanced, and the adaptability of the model to complex data can be enhanced.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and deep learning technology, and more specifically to a deep fuzzy clustering method for lung medical images based on coarse-fine granular contrastive learning. Background Technology

[0002] Lung cancer is one of the leading causes of cancer death worldwide, often accompanied by symptoms such as cough, shortness of breath, chest pain, and weight loss. In its early stages, it may not present with obvious symptoms, causing many patients to miss the optimal treatment window. Because different types of lung cancer exhibit significant differences in pathological characteristics and treatment response, accurately identifying the type, location, and biological behavior of the tumor is crucial for developing individualized treatment plans. In recent years, the application of deep learning technology has provided new opportunities for lung medical image analysis, particularly in the early diagnosis and subtype classification of lung cancer. Automated cluster analysis of lung medical images can achieve precise classification of different types of tumors. This not only improves the accuracy and efficiency of diagnosis but also assists doctors in comprehensively assessing tumor growth patterns, further promoting the development of personalized treatment and improving the prognosis of lung cancer patients.

[0003] The paper "An Intelligent Classification System for Lung Cancer Histopathological Images" proposes an intelligent image classification method based on a multi-classifier. However, this method's inability to effectively handle unlabeled data limits its application in unsupervised or weakly supervised learning scenarios. Furthermore, lung medical images are typically characterized by high dimensionality, unstructured nature, and noise interference, requiring more robust feature representation methods for subsequent clustering processes. Therefore, a novel method is urgently needed to process unlabeled data and efficiently acquire feature representations. This is crucial for improving the efficiency and accuracy of lung medical image analysis and advancing research applications of medical images in unlabeled data scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a deep fuzzy clustering method for lung medical images based on coarse-fine granular contrast learning. This method obtains feature representations of lung medical images and generates preliminary pseudo-labels through clustering, which helps improve the quality of pseudo-labels. By using a self-attention fuzzy online network and a self-attention fuzzy target network to obtain fuzzy feature representations, it helps to enhance the extraction of detailed features. By combining cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss, and mean squared error loss to optimize feature representations, it helps to improve the discriminative power of feature representations. This invention will provide important medical imaging evidence for the early diagnosis and subtype classification of lung cancer.

[0005] To achieve the above-mentioned objectives, the present invention employs the following technical solution: a deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrastive learning, comprising the following steps:

[0006] S10. Obtain feature representations of lung medical images and cluster them to generate preliminary pseudo-labels;

[0007] S20. Data augmentation to obtain enhanced images of lung medical images;

[0008] S30. Obtain fuzzy feature representations using a self-attention fuzzy online network and a self-attention fuzzy target network, respectively;

[0009] S40. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss, and mean squared error loss to optimize feature representation;

[0010] S50. Based on the preliminary clustering results and optimized feature representations of the current network, the clustering process is repeated, the clustering results and pseudo-labels are adjusted, and the pseudo-labels are used to guide the subsequent self-attention fuzzy feature representation learning; the clustering results and target network parameters are iteratively updated to obtain the final lung medical image clustering results.

[0011] As a deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrast learning provided by the present invention, step S10 includes the following steps:

[0012] S11. Input a batch of lung medical images X∈R N×H×W Where N is the number of samples in the current batch, H and W are the height and width of the lung medical image, respectively, and X contains the visual features of all images in the batch. A self-attention blurred target network is used to obtain the feature matrix. d1 is the feature dimension after the self-attention blurred target (online) network, with each row representing the feature of a sample;

[0013] S12. The obtained feature matrix Z is further updated through the target projector network g2(·) to obtain the updated feature matrix. d2 is the feature dimension obtained after passing through the projector network g2(·);

[0014] S13. Obtain the feature matrix of all samples in the current dataset using S12, perform fuzzy c-means clustering on the feature matrix of all samples, and use the clustering result of each sample as a pseudo-label. To guide subsequent self-attention fuzzy feature representation learning, where N all It represents the total number of samples in the lung medical image dataset.

[0015] As a deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrast learning provided by the present invention, step S20 includes the following steps:

[0016] S21. The enhancement methods for lung medical images are set. The deep blur clustering method for lung medical images based on coarse-grained contrast learning employs various data augmentation techniques, including random cropping, flipping, color enhancement, and Gaussian blur, as detailed below:

[0017] Random cropping involves randomly cropping an image and resizing it to a specified size. First, a random scale ∈ (s) is selected for a region. min ,s max Then, the area is cropped, where s min and s max It is the minimum and maximum ratio of the area of ​​the cropped region to the area of ​​the original image, and finally it is scaled to the specified target size;

[0018] Flipping is a horizontal rotation of an image with a 50% probability, changing the pixel position from (x, y) to (W). p -x,y), where (x,y) are the coordinates, W p It is the width of the image;

[0019] Color enhancement applies a color enhancement operation randomly with an 80% probability. It is achieved by adjusting the brightness, contrast, saturation, and hue of the image.

[0020] Gaussian blur applies Gaussian blur with a 50% probability. The convolution kernel used in Gaussian blur is a two-dimensional Gaussian distribution, represented as:

[0021]

[0022] Wherein, G(a,b,σ) b ) is a two-dimensional Gaussian distribution of the Gaussian blur convolution kernel, a and b are the offsets of each pixel in the image relative to the center, and σ is the offset of each pixel relative to the center. b It is the standard deviation of the Gaussian kernel, which controls the degree of ambiguity;

[0023] S22. Convert the enhanced lung medical image into a tensor, and then standardize it to a format suitable for deep learning models. The standardization formula is as follows:

[0024]

[0025] Where, x p ' is the normalized pixel value of the image channel, x p It is the pixel value of each image channel, μ n It is the mean of the image channels, σ n It is the standard deviation of the image channels;

[0026] S23. Using the above-described random enhancement settings, perform two random enhancements. Input the original version of the lung medical image X∈R for the current batch.N×H×W Two sets of two different enhanced versions of lung medical images were obtained. 1 ∈R N×H×W and X 2 ∈R N×H×W .

[0027] As a deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrast learning provided by the present invention, step S30 includes the following steps:

[0028] S31. Use a self-attention blurring online network and a self-attention blurring target network to extract two different enhanced versions of lung medical images X-rays. 1 and X 2 Visual features, to obtain the feature matrix and Where N is the number of samples in the current batch, and d1 is the feature dimension after passing through the self-attention blurred target (online) network:

[0029] Z 1 '=encoder online (X 1 (3)

[0030] Z 2 '=encoder target (X 2 (4)

[0031] Among them, X 1 and X 2 These are image enhancement and encoder. online (·) is a self-attention fuzzy online encoder network, encoder target (·) is a self-attention fuzzy target encoder network, Z 1 'and Z 2 ' is the obtained feature matrix;

[0032] S32. For the characteristic matrix A self-attention mechanism is employed, which weights and adjusts the feature representation of each sample by calculating the relationships between samples in the data. The input feature matrix Z... 1 'Mapped to query, key, and value spaces, the mapping process is represented as:'

[0033] Q = Z 1 'W Q (5)

[0034] K = Z 1 'W K (6)

[0035] V = Z 1 'W V (7)

[0036] Among them, Z 1 ' is the input feature matrix, It is a query matrix. It is a key matrix. It is a value matrix, W Q W K and W V These are the weight matrices learned by mapping to the query, key, and value spaces, respectively.

[0037] S33. Calculate the similarity between the query matrix and the key matrix using the dot product, and obtain the attention weights as follows:

[0038]

[0039] Where A∈R N×N It is a similarity matrix, where d1 is the feature dimension after passing through the self-attention blurred target (online) network; the similarity matrix A is normalized using the softmax function to obtain the attention score represented as A'∈R. N×N :

[0040] A' = softmax(A) (9)

[0041] Here, softmax(·) is an activation function; the value matrix V is weighted using attention scores A', and the process is expressed as follows:

[0042]

[0043] in, It is the weighted feature matrix;

[0044] S34. The obtained weighted characteristic matrix Fuzzy processing is performed, and M membership functions are applied to each feature point. The membership degree of each feature is calculated using these membership functions. The membership function used in this method is a Gaussian function, defined as follows:

[0045]

[0046] in, It is the weighted feature value of the i-th sample in the first enhanced version's feature space, in the j-th dimension, μ. k and σ k These are the mean and standard deviation of the k-th Gaussian membership function, respectively, μ k and σ k It is a trainable parameter, the membership function parameter μ for all feature points of the same sample. k and σ k They are the same;

[0047] S35. Apply fuzzy rules to aggregate the membership matrix and obtain the fuzzy feature value for each feature point:

[0048]

[0049] in, Z' is the weighted feature value of the i-th sample in the j-th dimension, where "Π" is the multiplication symbol. The fuzzy feature values ​​are processed by batch normalization to constrain their dynamic range. Then, an addition operation is used to integrate the fuzzy feature values ​​with the original feature Z', resulting in the final fuzzy feature matrix. The obtained feature matrix is ​​passed through an online projector network g1(·) to obtain the updated feature matrix.

[0050] S36, Enhanced medical image of the lungs (X-ray) 1 After passing through a self-attention fuzzy online network, the fuzzy feature matrix Z is obtained. 1 The updated fuzzy feature matrix g1(Z) is obtained through the online projector network g1(·). 1 The process is as described in S31-35, and involves the fuzzy feature matrix Z. 1 The acquisition process is the same, for lung medical image enhancement images X. 2 After passing through a self-attention fuzzy target network, the fuzzy feature matrix is ​​obtained. The updated fuzzy feature matrix is ​​obtained through the target projector network g2(·).

[0051] As a deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrast learning provided by the present invention, step S40 includes the following steps:

[0052] S41. For the feature matrix g1(Z) obtained in the above steps 1 ) and g2(Z 2 ), obtain K prototypes from an enhanced feature space. Another enhanced version of the feature space also obtained K prototypes. The prototype calculation formula for each cluster in the feature space of the two enhanced versions of each batch B is as follows:

[0053]

[0054] Where v is 1 or 2, representing the feature space of the two enhanced versions, and m = 1, 2, ..., K, representing K prototypes. It is the feature vector of the m-th prototype in the feature space of the v-th enhanced version. It is the feature vector of the i-th sample in the feature space of the v-th enhanced version. It is the fuzzy feature representation of the i-th sample in the feature space of the v-th enhanced version, u im It is the probability that the i-th sample belongs to the m-th cluster, and ||·||2 is the L2 norm, used to calculate the length of the vector;

[0055] S42. Using the prototype obtained from the above calculations, calculate the cluster-level coarse-grained contrast loss, expressed by the formula:

[0056]

[0057]

[0058]

[0059] in, and These are the feature vectors of the m-th prototype in the feature space of the first and second enhanced versions, respectively. and These are the feature vectors of the j-th prototype in the feature space of the first and second enhanced versions, respectively. "·" represents the vector dot product operation, τ c It is a temperature coefficient used to control the scaling factor in similarity calculation. and These are the cluster-level coarse-grained contrastive losses computed in the feature spaces of the first and second enhanced versions, respectively, L. c It is the overall cluster-level coarse-grained contrast loss;

[0060] S43. Calculate the sample-level fine-grained contrast loss, expressed by the formula:

[0061]

[0062]

[0063]

[0064] in, and These are the feature vectors of the i-th sample in the feature spaces of the first and second enhanced versions, respectively. It is relative to For all positive samples in the feature space of the second enhanced version, It is relative to For all positive samples in the feature space of the second enhanced version, z n It refers to all samples except the i-th sample, τ sHere, A(i) is the temperature coefficient, A(i) is the set of all samples in the feature space of the two enhanced versions, P(i) is all positive samples corresponding to the current sample, |P(i)| is the number of all positive samples corresponding to the current sample, and N is the number of samples in the current batch. and These are the sample-level coarse-grained contrastive losses computed in the feature spaces of the first and second enhanced versions, respectively, L. s It is the overall cluster-level coarse-grained contrast loss;

[0065] S44, the characteristic matrix g1(Z) 1 The predicted feature matrix f(g1(Z) is obtained after passing through the predictor network f(·). 1 The self-attention blurring online network is designed to predict the representation of the same image but with different augmentations by a self-attention blurring target network. The mean squared error loss L of the feature matrix mapped by the self-attention blurring online network is calculated. mse Represented as:

[0066]

[0067] Where f(·) is the predictor network, It is the square of the L2 norm;

[0068] S45. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss, and mean squared error loss to optimize the feature representation described in step S30. Update the parameters of the self-attention fuzzy online network through backpropagation and update the self-attention fuzzy target network through momentum.

[0069] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0070] (1) This invention generates preliminary pseudo-labels by performing feature representation and clustering on lung medical images. This process can guide subsequent feature representation learning, avoids dependence on a large amount of labeled data, and significantly improves the efficiency and effectiveness of unsupervised learning. The pseudo-labels not only guide subsequent feature optimization, but also provide a reliable initial framework for further data augmentation and fuzzy feature representation.

[0071] (2) This invention introduces a self-attention fuzzy feature representation module, which represents image features through a self-attention fuzzy online network and a self-attention fuzzy target network. This innovative method enables the model to better capture the relationship between image details and global information when processing lung medical images. By optimizing the fuzzy features, the model can more accurately capture subtle changes in key areas of the image, improving the accuracy of image clustering.

[0072] (3) The invention optimizes the feature representation training process by combining cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss, and mean squared error loss. The coarse-grained contrast loss optimizes the overall relationship between image clusters, while the fine-grained contrast loss focuses on the detailed features within the image, thereby improving the model's learning ability at different levels. This further enhances the reliability and accuracy of medical image analysis. Attached Figure Description

[0073] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0074] Figure 1 This is a patent flowchart of the lung medical image deep fuzzy clustering method based on coarse and fine granularity contrast learning of the present invention.

[0075] Figure 2 This is a patent framework diagram of the lung medical image deep fuzzy clustering method based on coarse and fine granularity contrast learning of the present invention.

[0076] Figure 3 This is a framework diagram of the self-attention fuzzy feature representation of the deep fuzzy clustering method for lung medical images based on coarse and fine granularity contrast learning according to the present invention.

[0077] Figure 4 This is a partial result diagram of Example 1 of the lung medical image deep fuzzy clustering method based on coarse and fine granularity contrast learning of the present invention.

[0078] Figure 5 This is a partial result diagram of Example 2 of the lung medical image deep fuzzy clustering method based on coarse and fine granularity contrast learning of the present invention. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0080] Example 1

[0081] See Figure 1-5 As shown, this embodiment provides a technical solution for a deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrastive learning, comprising the following steps:

[0082] S10. Obtain feature representations of lung medical images and cluster them to generate preliminary pseudo-labels;

[0083] S20. Obtain enhanced images of lung medical images using data augmentation techniques;

[0084] S30. Obtain fuzzy feature representations using a self-attention fuzzy online network and a self-attention fuzzy target network, respectively;

[0085] S40. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss, and mean squared error loss to optimize feature representation;

[0086] S50. Iteratively update the clustering results and the parameters of the self-attention fuzzy target network to obtain the final lung medical image clustering results.

[0087] Step S10 includes the following steps:

[0088] S11. Taking the lung CT scan image dataset from Kaggle as an example, this dataset contains four types of lung medical images, including pancreatic cancer, large cell carcinoma, squamous cell carcinoma, and normal tissue, totaling 1000 images. The input is a batch of lung medical images X∈R. 128×224×224 One batch contains 128 lung medical images, with a height and width of 224 and 224, respectively. X contains the visual features of all images in this batch. A self-attention blurred target network is used to obtain the feature matrix Z∈R. 128×512 The feature dimension after passing through the self-attention blurred target (online) network is 512, and each row represents the feature of a sample. X is shown in formula (1):

[0089]

[0090] S12. The obtained feature matrix Z is further updated through the target projector network g2(·), resulting in the updated feature matrix g2(Z)∈R. 128×256 d2 is the feature dimension obtained through the projector network g2(·), and g2(Z) is shown in Equation (2):

[0091]

[0092] S13. Obtain the feature matrix of all samples in the current dataset using S12, perform fuzzy c-means clustering on the feature matrix of all samples, and use the clustering result of each sample as a pseudo-label L. pseudo ∈R 1000×1 This guides subsequent self-attention fuzzy feature representation learning, where 1000 is the total number of samples in the lung medical image dataset, and L... pseudo As shown in formula (3):

[0093]

[0094] Step S20 includes the following steps:

[0095] S21. Set the enhancement method for lung medical images. The deep blur clustering method for lung medical images based on coarse and fine granularity contrast learning adopts a variety of data augmentation techniques, including random cropping, flipping, color enhancement, and Gaussian blur.

[0096] S22. Convert the enhanced lung medical images into tensors and then standardize them to a format suitable for deep learning models.

[0097] S23. Using the above-described random enhancement settings, perform two random enhancements. Input the original version of the lung medical image X∈R for the current batch. 128×224×224 Two sets of enhanced lung medical images X-rays were obtained. 1 ∈R 128×224×224 and X 2 ∈R 128 ×224×224 ,X 1 and X 2 As shown in formulas (4) and (5):

[0098]

[0099]

[0100] Step S30 includes the following steps:

[0101] S31. Use a self-attention blurring online network and a self-attention blurring target network to extract two different enhanced versions of lung medical images X-rays. 1 and X 2 Visual features, to obtain the feature matrix and Where N is the number of samples in the current batch, and d1 is the feature dimension after passing through the self-attention blurred target (online) network:

[0102] Z 1 '=encoder online (X 1 (6)

[0103] Z 2 '=encoder target (X 2 (7)

[0104]

[0105]

[0106] Among them, X 1 and X 2 These are image enhancement and encoder. online(·) is a self-attention fuzzy online encoder network, encoder target (·) is a self-attention fuzzy target encoder network, Z 1 'and Z 2 ' is the obtained feature matrix;

[0107] S32. For the characteristic matrix A self-attention mechanism is employed, which weights and adjusts the feature representation of each sample by calculating the relationships between samples in the data. The input feature matrix Z... 1 'Mapped to query, key, and value spaces, the mapping process is represented as:'

[0108] Q = Z 1 'W Q (10)

[0109]

[0110] K = Z 1 'W K (12)

[0111]

[0112] V = Z 1 'W V (14)

[0113]

[0114] Among them, Z 1 ' is the input feature matrix, It is a query matrix. It is a key matrix. It is a value matrix, W Q W K and W V These are the weight matrices learned by mapping to the query, key, and value spaces, respectively.

[0115] S33. Calculate the similarity between the query matrix and the key matrix using the dot product, and obtain the attention weights as follows:

[0116]

[0117]

[0118] Where A∈R N×N It is a similarity matrix, where d1 is the feature dimension after passing through the self-attention blurred target (online) network; the similarity matrix A is normalized using the softmax function to obtain the attention score represented as A'∈R. N×N :

[0119] A' = softmax(A) (18)

[0120]

[0121] Here, softmax(·) is an activation function; the value matrix V is weighted using attention scores A', and the process is expressed as follows:

[0122]

[0123]

[0124] in, It is the weighted feature matrix;

[0125] S34. The obtained weighted characteristic matrix Fuzzy processing is performed, and M membership functions are applied to each feature point. The membership degree of each feature is calculated using these membership functions. The membership function used in this method is a Gaussian function, defined as follows:

[0126]

[0127]

[0128] in, It is the weighted feature value of the i-th sample in the first enhanced version's feature space, in the j-th dimension, μ. k and σ k These are the mean and standard deviation of the k-th Gaussian membership function, respectively, μ k and σ k It is a trainable parameter, the membership function parameter μ for all feature points of the same sample. k and σ k They are the same;

[0129] S35. Apply fuzzy rules to aggregate the membership matrix and obtain the fuzzy feature value for each feature point:

[0130]

[0131]

[0132] in, Z' is the weighted feature value of the i-th sample in the j-th dimension, where "Π" is the multiplication symbol. The fuzzy feature values ​​are processed by batch normalization to constrain their dynamic range. Then, an addition operation is used to integrate the fuzzy feature values ​​with the original feature Z', resulting in the final fuzzy feature matrix. The obtained feature matrix is ​​passed through an online projector network g1(·) to obtain the updated feature matrix.

[0133] S36, Enhanced medical image of the lungs (X-ray) 1 After passing through a self-attention fuzzy online network, the fuzzy feature matrix Z is obtained. 1 The updated fuzzy feature matrix g1(Z) is obtained through the online projector network g1(·). 1 The process is as described in S31-35, and involves the fuzzy feature matrix Z. 1 The acquisition process is the same, for lung medical image enhancement images X. 2 After passing through a self-attention fuzzy target network, the fuzzy feature matrix is ​​obtained. The updated fuzzy feature matrix is ​​obtained through the target projector network g2(·).

[0134] Step S40 includes the following steps:

[0135] S41. For the feature matrix g1(Z) obtained in the above steps 1 ) and g2(Z 2 ), obtain 4 prototypes from an enhanced feature space. Another enhanced version of the feature space also obtained 4 prototypes. The prototype calculation formula for each cluster in the feature space of the two enhanced versions of each batch B is as follows:

[0136]

[0137]

[0138]

[0139] Where v is 1 or 2, representing the feature space of the two enhanced versions, and m = 1, 2, ..., K, representing K prototypes. It is the feature vector of the m-th prototype in the feature space of the v-th enhanced version. It is the feature vector of the i-th sample in the feature space of the v-th enhanced version. It is the fuzzy feature representation of the i-th sample in the feature space of the v-th enhanced version, u im It is the probability that the i-th sample belongs to the m-th cluster, and ||·||2 is the L2 norm, used to calculate the length of the vector;

[0140] S42. Using the prototype obtained from the above calculations, calculate the cluster-level coarse-grained contrast loss, expressed by the formula:

[0141]

[0142]

[0143]

[0144] in, and These are the feature vectors of the m-th prototype in the feature space of the first and second enhanced versions, respectively. and These are the feature vectors of the j-th prototype in the feature space of the first and second enhanced versions, respectively. "·" represents the vector dot product operation, τ c It is a temperature coefficient used to control the scaling factor in similarity calculation. and These are the cluster-level coarse-grained contrastive losses computed in the feature spaces of the first and second enhanced versions, respectively, L. c It is the overall cluster-level coarse-grained contrast loss;

[0145] S43. Calculate the sample-level fine-grained contrast loss, expressed by the formula:

[0146]

[0147]

[0148] in, and These are the feature vectors of the i-th sample in the feature spaces of the first and second enhanced versions, respectively. It is relative to For all positive samples in the feature space of the second enhanced version, It is relative to For all positive samples in the feature space of the second enhanced version, z n It refers to all samples except the i-th sample, τ s Here, A(i) is the temperature coefficient, A(i) is the set of all samples in the feature space of the two enhanced versions, P(i) is all positive samples corresponding to the current sample, |P(i)| is the number of all positive samples corresponding to the current sample, and N is the number of samples in the current batch. and These are the sample-level coarse-grained contrastive losses computed in the feature spaces of the first and second enhanced versions, respectively, L. s It is the overall cluster-level coarse-grained contrast loss;

[0149] S44, the characteristic matrix g1(Z) 1 The predicted feature matrix f(g1(Z) is obtained after passing through the predictor network f(·). 1The self-attention blurring online network is designed to predict the representation of the same image but with different augmentations by a self-attention blurring target network. The mean squared error loss L of the feature matrix mapped by the self-attention blurring online network is calculated. mse Represented as:

[0150]

[0151] Where f(·) is the predictor network, It is the square of the L2 norm;

[0152] S45. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss, and mean squared error loss to optimize the feature representation described in step S30. Update the parameters of the self-attention fuzzy online network through backpropagation and update the self-attention fuzzy target network through momentum.

[0153] Step S50 includes the following steps:

[0154] S51. Based on the preliminary clustering results and optimized feature representations of the current network, the clustering process is repeated, the clustering results and pseudo-labels are adjusted, and the pseudo-labels are used to guide the subsequent self-attention fuzzy feature representation learning; the clustering results and target network parameters are iteratively updated to obtain the final lung medical image clustering results.

[0155] The lung CT scan image dataset was input into three different models for clustering tasks: the first two were traditional clustering models, and the third was our proposed method. The clustering accuracy for all samples was calculated. True indicates the number of samples correctly clustered by the model, and False indicates the number of samples correctly clustered by the model. The results show that the clustering results of this method are good.

[0156] Table 1

[0157]

[0158] Example 2

[0159] See Figure 1-5 As shown, this embodiment provides a technical solution for a deep fuzzy clustering method for lung medical images based on coarse-fine granularity contrastive learning, comprising the following steps:

[0160] S10. Obtain feature representations of lung medical images and cluster them to generate preliminary pseudo-labels;

[0161] S20. Obtain enhanced images of lung medical images using data augmentation techniques;

[0162] S30. Obtain fuzzy feature representations using a self-attention fuzzy online network and a self-attention fuzzy target network, respectively;

[0163] S40. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss, and mean squared error loss to optimize feature representation;

[0164] S50. Iteratively update the clustering results and the parameters of the self-attention fuzzy target network to obtain the final lung medical image clustering results.

[0165] Step S10 includes the following steps:

[0166] S11. Taking the X-ray image pneumonia dataset from Kaggle as an example, this dataset contains three types of lung medical images: normal, viral pneumonia, and bacterial pneumonia, totaling 5284 images. The input is a batch of lung medical images X∈R. 128×224×224 One batch contains 128 lung medical images, with a height and width of 224 and 224, respectively. X contains the visual features of all images in this batch. A self-attention blurred target network is used to obtain the feature matrix Z∈R. 128×512 The feature dimension after passing through the self-attention blurred target (online) network is 512, and each row represents the feature of a sample. X is shown in formula (1):

[0167]

[0168] S12. The obtained feature matrix Z is further updated through the target projector network g2(·), resulting in the updated feature matrix g2(Z)∈R. 128×256 d2 is the feature dimension obtained through the projector network g2(·), and g2(Z) is shown in Equation (2):

[0169]

[0170] S13. Obtain the feature matrix of all samples in the current dataset using S12, perform fuzzy c-means clustering on the feature matrix of all samples, and use the clustering result of each sample as a pseudo-label L. pseudo ∈R 1000×1 This guides subsequent self-attention fuzzy feature representation learning, where 5284 is the total number of samples in the lung medical image dataset, and L... pseudo As shown in formula (3):

[0171]

[0172] Step S20 includes the following steps:

[0173] S21. Set the enhancement method for lung medical images. The deep blur clustering method for lung medical images based on coarse and fine granularity contrast learning adopts a variety of data augmentation techniques, including random cropping, flipping, color enhancement, and Gaussian blur.

[0174] S22. Convert the enhanced lung medical images into tensors and then standardize them to a format suitable for deep learning models.

[0175] S23. Using the above-described random enhancement settings, perform two random enhancements. Input the original version of the lung medical image X∈R for the current batch. 128×224×224 Two sets of enhanced lung medical images X-rays were obtained. 1 ∈R 128×224×224 and X 2 ∈R 128 ×224×224 ,X 1 and X 2 As shown in formulas (4) and (5):

[0176]

[0177]

[0178] Step S30 includes the following steps:

[0179] S31. Use a self-attention blurring online network and a self-attention blurring target network to extract two different enhanced versions of lung medical images X-rays. 1 and X 2 Visual features, to obtain the feature matrix and Where N is the number of samples in the current batch, and d1 is the feature dimension after passing through the self-attention blurred target (online) network:

[0180]

[0181]

[0182] S32. For the characteristic matrix A self-attention mechanism is employed, which weights and adjusts the feature representation of each sample by calculating the relationships between samples in the data. The input feature matrix Z... 1 'Mapped to query, key, and value spaces, the mapping process is represented as:'

[0183]

[0184]

[0185]

[0186] S33. Calculate the similarity between the query matrix and the key matrix using the dot product, and obtain the attention weights as follows:

[0187]

[0188] The similarity matrix A is normalized using the softmax function to obtain the attention score, represented as A'∈R. N×N :

[0189]

[0190] The value matrix V is weighted using attention scores A', and the process is expressed as follows:

[0191]

[0192] S34. The obtained weighted characteristic matrix Fuzzy processing is performed, and M membership functions are applied to each feature point. The membership degree of each feature is calculated using these membership functions. The membership function used in this method is a Gaussian function.

[0193]

[0194] S35. Apply fuzzy rules to aggregate the membership matrix and obtain the fuzzy feature value for each feature point:

[0195]

[0196] The fuzzy feature values ​​are processed by batch normalization to constrain their dynamic range. Then, an addition operation is used to integrate the fuzzy feature values ​​with the original feature Z' to obtain the final fuzzy feature matrix. The obtained feature matrix is ​​passed through an online projector network g1(·) to obtain the updated feature matrix.

[0197] S36, Enhanced medical image of the lungs (X-ray) 1 After passing through a self-attention fuzzy online network, the fuzzy feature matrix Z is obtained. 1 The updated fuzzy feature matrix g1(Z) is obtained through the online projector network g1(·). 1 The process is as described in S31-35, and involves the fuzzy feature matrix Z. 1 The acquisition process is the same, for lung medical image enhancement images X. 2 After passing through a self-attention fuzzy target network, the fuzzy feature matrix is ​​obtained. The updated fuzzy feature matrix is ​​obtained through the target projector network g2(·).

[0198] Step S40 includes the following steps:

[0199] S41. For the feature matrix g1(Z) obtained in the above steps 1 ) and g2(Z 2 ), obtain 3 prototypes from an enhanced feature space. Another enhanced version of the feature space also yielded three prototypes. The prototype calculation formula for each cluster in the feature space of the two enhanced versions of each batch B is as follows:

[0200]

[0201]

[0202] S42. Use the prototype obtained from the above calculation to calculate the cluster-level coarse-grained contrast loss.

[0203] S43. Calculate the sample-level fine-grained contrast loss.

[0204] S44. Calculate the mean squared error loss of the feature matrix of the prediction and target network mapping of the self-attention fuzzy online network.

[0205] S45. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss, and mean squared error loss to optimize the feature representation described in step S30. Update the parameters of the self-attention fuzzy online network through backpropagation and update the self-attention fuzzy target network through momentum.

[0206] Step S50 includes the following steps:

[0207] S51. Iteratively update the clustering results and target network parameters to obtain the final lung medical image clustering results.

[0208] The X-ray image pneumonia dataset was input into three different models for clustering tasks: the first two were traditional clustering models, and the third was our proposed method. The clustering accuracy for all samples was calculated.

[0209] Table 2

[0210]

[0211] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deep fuzzy clustering method for lung medical images based on coarse-fine granular contrastive learning, characterized in that, Includes the following steps: S10. Obtain feature representations of lung medical images and cluster them to generate preliminary pseudo-labels; Step S10 includes the following steps: S11. Input a batch of lung medical images. Where N is the number of samples in the current batch, and H and W are the height and width of the lung medical image, respectively. The feature matrix, containing visual features of all images in the batch, is obtained using a self-attention blurred target network. d1 is the feature dimension after the self-attention blurred target online network, and each row represents the feature of a sample; S12. Obtain the characteristic matrix Through target projector network Further update the feature matrix to obtain the updated feature matrix. d2 is through the projector network The subsequent feature dimensions; S13. Obtain the feature matrix of all samples in the current dataset through step S12, perform fuzzy c-means clustering on the feature matrix of all samples, and use the clustering result of each sample as a pseudo-label. To guide subsequent self-attention fuzzy feature representation learning, where N all This represents the total number of samples in the lung medical image dataset. S20. Obtain enhanced images of lung medical images using data augmentation techniques; S30. Obtain fuzzy feature representations using a self-attention fuzzy online network and a self-attention fuzzy target network, respectively; S40. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss, and mean squared error loss to optimize feature representation; S50. Iteratively update the clustering results and the parameters of the self-attention fuzzy target network to obtain the final lung medical image clustering results.

2. The method for deep fuzzy clustering of lung medical images based on coarse-grained contrastive learning according to claim 1, characterized in that, Step S20 includes the following steps: S21. The enhancement methods for lung medical images are set. The deep blur clustering method for lung medical images based on coarse-grained contrast learning employs various data augmentation techniques, including random cropping, flipping, color enhancement, and Gaussian blur, as detailed below: Random cropping randomly crops an image and resizes it to a specified size. First, a random proportion of a selected area is chosen. Then, the area is cropped, where and It is the minimum and maximum ratio of the area of ​​the cropped region to the area of ​​the original image, and finally it is scaled to the specified target size; Flipping is a horizontal rotation of an image with a 50% probability, changing the pixel position from (x, y) to (W). p -x, y), where (x, y) are the coordinates, W p It is the width of the image; Color enhancement applies a color enhancement operation randomly with an 80% probability. It is achieved by adjusting the brightness, contrast, saturation, and hue of the image. Gaussian blur applies Gaussian blur with a 50% probability. The convolution kernel used in Gaussian blur is a two-dimensional Gaussian distribution, represented as: (1) in, It is a two-dimensional Gaussian distribution of the Gaussian blur convolution kernel, where a and b are the offsets of each pixel in the image relative to the center. It is the standard deviation of the Gaussian kernel, which controls the degree of ambiguity; S22. Convert the enhanced lung medical image into a tensor, and then standardize it to a format suitable for deep learning models. The standardization formula is as follows: (2) in, It is the standardized pixel value of the image channel, x p It is the pixel value of each image channel. It is the average value of the image channels. It is the standard deviation of the image channels; S23. Using the above-described random enhancement settings, perform two random enhancements, and input the original version of the lung medical image for the current batch. Two sets of two different enhanced versions of lung medical images were obtained. and .

3. The method for deep fuzzy clustering of lung medical images based on coarse-grained and fine-grained contrastive learning according to claim 1, characterized in that, Step S30 includes the following steps: S31. Use a self-attention blurring online network and a self-attention blurring target network to extract two different enhanced versions of lung medical images. and Visual features, to obtain the feature matrix and Where N is the number of samples in the current batch, and d1 is the feature dimension after the target is blurred by the self-attention online network: (3) (4) in, and These are image enhancement, It is a self-attention fuzzy online encoder network. It is a self-attention fuzzy target encoder network. and It is the feature matrix obtained; S32. For the characteristic matrix It employs a self-attention mechanism, weighting and adjusting the feature representation of each sample by calculating the relationships between samples in the data, and inputting the feature matrix. The mapping process, which involves mapping to query, key, and value spaces, can be represented as follows: (5) (6) (7) in, It is the input feature matrix, Q It is a query matrix, K It is a key matrix, V It is a value matrix. , and These are the weight matrices learned by mapping to the query, key, and value spaces, respectively. S33. Calculate the similarity between the query matrix and the key matrix using the dot product, and obtain the attention weights as follows: (8) Among them, A It is a similarity matrix, where d1 is the feature dimension after the target is blurred through the self-attention online network; the similarity matrix A is normalized using the softmax function to obtain the attention score. : (9) in, It is an activation function; it utilizes attention scores. The weighting of the value matrix V is represented as follows: (10) in, It is the weighted feature matrix; S34. The obtained weighted characteristic matrix , , The method performs fuzzy processing, applying M membership functions to each feature point. The membership degree of each feature is calculated using these membership functions. The membership function used is a Gaussian function, defined as: (11) in, , , , It is the weighted feature value of the i-th sample in the j-th dimension of the feature space of the first enhanced version. and These are the mean and standard deviation of the k-th Gaussian membership function, respectively. and These are trainable parameters, the membership function parameters for all feature points of the same sample. and They are the same; S35. Apply fuzzy rules to aggregate the membership matrix and obtain the fuzzy feature value for each feature point: (12) in, It is the weighted feature value of the i-th sample in the j-th dimension. " is the multiplication symbol; fuzzy feature values ​​are processed through batch normalization to constrain their dynamic range, and then addition is used to combine the fuzzy feature values ​​with the original feature values. The final fuzzy feature matrix Z is obtained by integration. 1 The obtained feature matrix is ​​then passed through an online projector network. The updated feature matrix is ​​obtained. ; S36, Enhanced medical images of the lungs After passing through a self-attention fuzzy online network, the fuzzy feature matrix Z is obtained. 1 via online projector network The updated fuzzy feature matrix is ​​obtained. The process is as described in steps S31 to S35, and involves the fuzzy feature matrix Z. 1 The acquisition process is the same, for enhanced images of lung medical images. After passing through a self-attention fuzzy target network, the fuzzy feature matrix Z is obtained. 2 Through target projector network The updated fuzzy feature matrix is ​​obtained. .

4. The method for deep fuzzy clustering of lung medical images based on coarse-grained and fine-grained contrastive learning according to claim 1, characterized in that, Step S40 includes the following steps: S41. Regarding the feature matrix obtained in the above steps and Obtain K prototypes from an enhanced feature space Another enhanced version of the feature space also obtained K prototypes. The prototype calculation formula for each cluster in the feature space of the two enhanced versions of each batch B is as follows: (13) Where v is 1 or 2, representing two enhanced versions of the feature space. , representing K prototypes, It is the feature vector of the m-th prototype in the feature space of the v-th enhanced version. It is the feature vector of the i-th sample in the feature space of the v-th enhanced version. It is the fuzzy feature representation of the i-th sample in the feature space of the v-th enhanced version. It is the probability that the i-th sample belongs to the m-th cluster. It is the L2 norm, used to calculate the length of a vector; S42. Using the prototype obtained from the above calculations, calculate the cluster-level coarse-grained contrast loss, expressed by the formula: (14) (15) (16) in, and These are the feature vectors of the m-th prototype in the feature space of the first and second enhanced versions, respectively. and These are the feature vectors of the j-th prototype in the feature space of the first and second enhanced versions, respectively. " represents the vector dot product operation, It is a temperature coefficient used to control the scaling factor in similarity calculation. and These are the cluster-level coarse-grained contrastive losses computed in the feature spaces of the first and second enhanced versions, respectively. It is the overall cluster-level coarse-grained contrast loss; S43. Calculate the sample-level fine-grained contrast loss, expressed by the formula: (17) (18) (19) in, and These are the feature vectors of the i-th sample in the feature spaces of the first and second enhanced versions, respectively. It is relative to For all positive samples in the feature space of the second enhanced version, It is relative to For all positive samples in the feature space of the second enhanced version, It refers to all samples except the i-th sample. It is the temperature coefficient. It is the set of all samples in the two enhanced versions of the feature space. It represents all positive samples corresponding to the current sample. This represents the number of all positive samples corresponding to the current sample, and N is the number of samples in the current batch. and These are the sample-level coarse-grained contrastive losses computed in the feature spaces of the first and second enhanced versions, respectively. It is the overall cluster-level coarse-grained contrast loss; S44, the feature matrix Through predictor network The predicted feature matrix is ​​then obtained. Self-attention blurring online networks are used to predict the representations of the same image but with different augmentations by a self-attention blurring target network. The mean squared error loss of the feature matrix mapped by the self-attention blurring online network is calculated. Represented as: (20) in, It is a predictor network. It is the square of the L2 norm; S45. Combine cluster-level coarse-grained contrast loss, sample-level fine-grained contrast loss, and mean squared error loss to optimize the feature representation described in step S30. Update the parameters of the self-attention fuzzy online network through backpropagation and update the self-attention fuzzy target network through momentum.

5. The lung medical image deep fuzzy clustering method based on coarse-fine granularity contrastive learning according to claim 1, in step S50, based on the preliminary clustering results and optimized feature representations of the current network, the clustering process is repeated, the clustering results and pseudo-labels are adjusted, and the pseudo-labels are used to guide the subsequent self-attention fuzzy feature representation learning; the clustering results and target network parameters are iteratively updated to obtain the final lung medical image clustering results.

Images