A deep multi-granularity granule contrast clustering method for melanoma diagnosis

Abstract

The application provides a deep multi-granularity granule contrast clustering method for melanoma diagnosis, comprising the following steps: step 1, pre-processing skin disease image data sets to generate two sample image data sets; step 2, performing feature extraction on the sample images by using a pre-trained deep neural network; step 3, training by using a multi-granularity granule contrast method; step 4, fusing features extracted in each stage; and step 5, performing image contrast clustering by using a self-encoder. The scheme automatically extracts features of skin disease images, accurately captures key features, improves the existing deep clustering effect, provides an important classification basis for the prediction of melanoma patients, and helps doctors to take treatment and rehabilitation measures in advance.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, and particularly relates to a deep multi-granularity granulocyte-sphere contrastive clustering method for melanoma diagnosis. Background Technology

[0002] Melanoma is a malignant tumor originating from melanocytes in the skin or mucous membranes. It is highly invasive and metastatic, making it one of the most dangerous types of skin cancer, with an increasing incidence worldwide. Early detection and timely treatment are crucial for melanoma patients. However, due to the complexity of skin lesions, symptoms of basal cell carcinoma and actinic keratosis are very similar to those of melanoma, greatly complicating its diagnosis. Regarding skin image data, traditional manual evaluation methods suffer from limitations such as strong subjectivity, low efficiency, and data bias when processing such high-dimensional data, resulting in insufficient accuracy in diagnosing melanoma patients. How to automatically integrate and analyze these complex features to provide accurate predictions for melanoma diagnosis, thereby assisting doctors in clinical decision-making, is a highly challenging issue. With the rapid development of technology, deep learning, due to its ability to process complex and high-dimensional data without human intervention, has significant application value in melanoma diagnosis.

[0003] Deep learning automatically learns useful features from large amounts of data by constructing multi-layered neural network models, enabling complex tasks. Clustering divides samples in a dataset into several clusters, making samples within the same cluster as similar as possible, while samples in different clusters are as different as possible. It is a method of automatically labeling data, and with its combination with deep learning, deep clustering has become a more efficient clustering method. Contrastive learning is one of the widely used techniques in deep clustering. It uses positive and negative samples to guide the deep learning model. The core of contrastive learning is to maximize the similarity between positive data pairs while minimizing the similarity between negative pairs.

[0004] Currently, Zhang et al., in "Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning" (Computer Vision–ECCV 2024, pp35-52), pointed out that in contrastive learning, the mining of latent features is insufficient, ignoring potentially valuable features, i.e., the feature suppression phenomenon. Zhang et al. proposed a feature-aware negative sampling method, constructing negative sample pairs through K-means clustering. However, K-means clustering requires specifying the number of clusters, which is not flexible enough for extracting features from datasets with different sample data distributions. Xie et al., in "Research on Efficient Fuzzy Clustering Method Based on Local Fuzzy Granular Balls" (arXiv preprint, arXiv:2303.03590, 2023), proposed a granular ball clustering algorithm that does not require setting the number of clusters and can adapt to the characteristics of sample data distribution. This algorithm merges granular balls according to overlapping relationships to form the final clustering result. However, this method lacks detailed discrimination of local structure and often suffers from over-aggregation when constructing granular ball overlapping relationships. Therefore, how to generate discriminative particle clustering results during the model learning process and fully explore the potential features of skin disease images is a challenge faced by existing technologies. Summary of the Invention

[0005] Objective: This invention addresses the shortcomings of existing technologies by providing a deep multi-granularity granulocyte-sphere contrastive clustering method for melanoma diagnosis. It extracts image features using a pre-trained deep neural network, then iteratively mines sample features using a multi-granularity granulocyte-sphere contrastive method. The proposed granulocyte-sphere clustering method adapts to sample pair distribution characteristics without requiring additional hyperparameters and avoids the over-aggregation problem found in existing methods. Finally, an autoencoder mapping is trained using the contrastive clustering method to obtain the final clustering result. This invention will provide a new method for deep feature extraction and image clustering of dermatological images, offering a more scientific basis for melanoma diagnosis.

[0006] The method of the present invention includes the following steps:

[0007] Step 1: Preprocess the skin disease image dataset to generate two sample image datasets;

[0008] Step 2: Use a pre-trained deep neural network to extract features from the two sample image datasets to obtain two sample feature sets.

[0009] Step 3: Use the multi-granularity particle-sphere comparison method for training. Construct negative samples by using particle-sphere clustering to extract features from the previous granularity layer, perform comparative learning, and use the extracted features for the next granularity layer. Repeat this process for training.

[0010] Step 4: Integrate the features extracted from each stage by concatenating the features from each stage of training element by element.

[0011] Step 5: Perform comparative clustering using an autoencoder. Add the two membership matrices obtained from the spliced ​​feature inputs to obtain the final clustering result.

[0012] In step 1, the skin disease image dataset was denoised and normalized, and then two data augmentation processes were performed to generate two enhanced sample image datasets. The two enhanced sample image datasets were then merged into a single dataset D. The data augmentation processes included random flipping, color transformation, and adding Gaussian noise of different intensities.

[0013] In step 2, a pre-trained deep neural network is used to extract features from dataset D, capturing high-level features of the input data to obtain a sample feature set. Pre-trained deep neural networks are usually trained on large datasets and have strong generalization and transfer learning capabilities.

[0014] Step 3 includes the following steps:

[0015] Step 3-1: The features of the sample feature set obtained in Step 2 are used as the features of the first granularity layer. Particle-sphere clustering is then performed on dataset D for negative sample sampling. A particle-sphere is a spherical structure covering a local sample, defined as a sphere with center c and radius r, used to represent a local region of the dataset. The calculation formula is:

[0016]

[0017] Where, x n It is the nth sample in the sphere, N b is the number of samples covered by the sphere, and ||.|| is the Euclidean norm;

[0018] Let the total number of samples in dataset D be N. First, construct a sphere from the entire dataset. Divide the sphere into two subspheres. Then, continue dividing each subsphere until the resulting sphere no longer meets the partitioning criteria. The final set of spheres is GB = {Ω1,Ω2,...,Ω} t}, where t represents the total number of particles in the particle set GB, Ω t This represents the t-th ball;

[0019] Let the sphere Ω covering the sample set be Y. YThe conditions for partitioning are met, and the particles are divided into two sub-spheres. and The specific division rules are as follows:

[0020]

[0021] Y α ={y i ∈Y,||y i -y α ||≤||y i -y β ||} (4),

[0022] Y β =YY α (5),

[0023] Where y i and y j Given any two samples in the sample set Y, use argmax(.) to find the two samples y that are farthest apart in the sample set Y. α and y β ;Y α Represents sub-spheres distributed by distance Covered sample set; Y β Represents sub-spheres distributed by distance The covered sample set;

[0024] Step 3-2: Based on the divided spheres, generate cluster assignments and construct negative sample pairs for the current granularity layer; let Ω be any two spheres in the sphere set GB. i and Ω j The centers of the spheres are c and c respectively. i and c j The radii are r i and r j If the following conditions are met:

[0025] ||c i -c j || <r i +r j +max(r i ,r j (6),

[0026] Then determine the two spheres Ω i and Ω j Overlapping, according to the conditions of formula (6), the particle clustering results are initially generated, and the particles with overlapping relationships are assigned to the same cluster. Since the overlap condition is set too broadly, it will lead to the clustering results being overly concentrated. Therefore, the abnormally assigned particles are detected by combining particle density and particle connection angle, and the clusters where the particles are located are further divided.

[0027] Let Ω be the particle. i Overlapping spheres are E i For OL i The number of particles in the middle, OL i Middle E i Individual spheres; Ω i The category is Q i For CL i The number of particles in the middle, OL i Q in the middle i Individual spheres; sphere Ω i The number of samples covered is n i , will OL i Any two spheres in the array are denoted as Ω. (i,ε) and Ω (i,ξ) ε≠ξ and 1≤ε≤E i , 1≤ξ≤E i , granules Ω i minimum overlap angle θ i Defined as:

[0028]

[0029] Among them, V ε Ω represents a granular particle i The center of the sphere points to the sphere Ω (i,ε) The vector of the center of the sphere, V ξ Ω represents a granular particle i The center of the sphere points to the sphere Ω (i,ξ) The vector of the center of the sphere; sphere Ω i density ρ i Defined as:

[0030]

[0031] If the granule Ω i The following conditions must be met:

[0032] and

[0033] Then determine the particle Ω i Cluster CL i Allocation error, CL i Further partitioning is needed; let θ i ε and ξ at the minimum value are respectively and Corresponding to OL i The granules in the middle are and CLi All the particles according to and The distance is further divided into 2 clusters. and Let CL i Any sphere in Ω [i,δ] , 1≤δ≤Q i The specific division rules are as follows:

[0034]

[0035] Where c δ , and Ω, respectively [i,δ] The center of the ball, The center of the ball and The center of the ball;

[0036] After performing step 3-2 detection on all spheres in the sphere set GB and reclassifying them into their corresponding clusters, assign the same label to samples covered by all spheres in the same cluster, setting the current label to the g-th sphere. c Grain size layer, g c ≥2 (excluding the first granularity layer, which is unlabeled), if any sample pair is in the second granularity layer to the gth granularity layer c If all granularity layers are assigned the same label, they constitute negative sample pairs for the current granularity layer;

[0037] Step 3-3: Based on the negative sample pairs of the second granularity layer, a contrastive learning method is used for the first stage of learning to obtain the feature representation of the second granularity layer; a multilayer perceptron (MLP) is used for feature mapping to maximize the similarity of negative sample pairs and minimize the similarity of positive sample pairs. The InfoNCE loss function of the i-th sample in dataset D is... Defined as:

[0038]

[0039] Where exp is the natural exponential function, z i It is the feature representation of the i-th sample after MLP mapping. It is z i The corresponding positive samples are represented by features after MLP mapping, where sim is the similarity function and τ is the similarity function. ins It is a temperature coefficient used to control the smoothness of the probability distribution, N neg It is the number of negative samples of the i-th sample, and the positive sample pair is the corresponding sample of two sample image datasets in dataset D;

[0040] The formula for calculating the similarity sim(z1,z2) between any two features z1 and z2 is:

[0041]

[0042] in Indicates transpose;

[0043] Add a regularization term to the InfoNCE loss function to control the similarity of negative sample pairs. The formula is:

[0044]

[0045] Complete objective function L ins Represented as:

[0046]

[0047] Where α is the balance coefficient of the regularization term;

[0048] Steps 3-4: Repeat steps 3-1 to 3-3 to perform the second stage of training and obtain the feature representation of the third granularity layer. Repeat this process for a total of S stages. The last stage of training will obtain the feature representation of the (S+1)th granularity layer.

[0049] In steps 3-4, S satisfies:

[0050]

[0051] Among them, K s B is the number of clusters in the s-th stage of particle-sphere clustering, and B is the batch size.

[0052] In step 4, the extracted features at each granularity level are concatenated element-wise. Let sample set a and sample set b be the two enhanced sample image datasets obtained in step 1, and let all sample features at the s-th granularity level of sample set v be... The final sample set v contains all the concatenated sample features h v for:

[0053]

[0054] Step 5 includes the following steps:

[0055] Step 5-1: Input the features extracted from the two sample image datasets into the autoencoder. The encoding and decoding layers of the autoencoder output the encoded and reconstructed feature data respectively. The intermediate layer is the embedding layer of the autoencoder. The dimension is set to the final number of clusters. The output of the embedding layer is converted into the membership degree of each sample to each cluster, that is, the probability of each sample belonging to each cluster. At the same time, the network structure of the autoencoder is optimized by minimizing the mean square error between the input and output of the features.

[0056] Step 5-2, the definition of an autoencoder is:

[0057] h = f(W1x + b1)(18),

[0058] y = g(W2h + b2) (19),

[0059] Where x is the input sample feature data, W1, W2 and b1, b2 are coefficient matrices, f(·) is the encoder of the autoencoder, h is the output of the encoder, g(·) is the decoder of the autoencoder, and y is the output of the decoder. Both the encoder and decoder are fully connected layers with two or more layers, and the encoder and decoder have symmetrical structures.

[0060] Step 5-3: Optimize the autoencoder parameters by minimizing the decoder's reconstruction error. The autoencoder's reconstruction error L... re Using the minimum mean square error, it is defined as:

[0061]

[0062] Where, q i The features are concatenated from the i-th sample in D after S stages of learning. It is q i The output reconstructed by the decoder;

[0063] Step 5-4: Let the dimension of the autoencoder embedding layer be the number of clusters C. The output of the embedding layer represents the membership degree of each sample to each cluster. After normalizing the output, any element in the membership degree matrix of the sample set v is... satisfy:

[0064]

[0065] Step 5-5: Perform comparative training on the two membership matrices obtained by concatenating the features of the two sample image datasets in dataset D. Let the membership vector of the j-th sample in sample set a belong to each cluster be... The membership vector of the j-th sample in sample set b to each cluster is: The loss function of the j-th sample in sample set a The loss function of the j-th sample in sample set b They are respectively:

[0066]

[0067] Where, τ clu It is a temperature coefficient that controls the smoothness of the distribution; the clustering results are compared with the training objective function L. clu Represented as:

[0068]

[0069] Where H(Y) is the entropy of the probability distribution, expressed as:

[0070]

[0071] in, The average probability assigned to the k-th cluster in the sample set v is expressed as:

[0072]

[0073] Steps 5-6: The objective function L for autoencoder training ae Represented as:

[0074] L ae =L re +L clu (27);

[0075] Steps 5-7: After training, the two membership matrices are added together to form the final assignment matrix μ. Any element μ in the assignment matrix μ... jk This represents the final probability of the j-th sample being assigned to the k-th cluster. The final clustering result is obtained based on the assignment matrix. That is, for each sample, the cluster with the highest value in the assignment matrix is ​​selected as the final cluster.

[0076] In steps 5-7, the final clustering result is represented by the following formula:

[0077]

[0078] Where Cluster(j) represents the clustering result of the j-th sample.

[0079] The present invention also provides an electronic device, including a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0080] The present invention also provides a storage medium storing a computer program or instructions that, when the computer program or instructions are run on a computer, execute the steps of the method described.

[0081] The present invention has the following beneficial effects: (1) The present invention uses a multi-granularity contrastive learning and fusion method, which can fully explore the potential features in the sample set and provide richer and more distinguishable feature representations for clustering tasks.

[0082] (2) Based on the idea of ​​particle-sphere clustering, this invention uses particle-sphere structure to construct negative sample pairs for contrastive learning. Compared with the K-means method, it does not require additional hyperparameters and can adapt to the distribution characteristics of the samples, thereby improving the accuracy of negative sampling.

[0083] (3) The present invention redesigns a parameter-free particle clustering method. First, the initial overlap constraint is relaxed to generate a preliminary clustering assignment. Then, based on the internal density of the particle and the overlap angle of the particle, the abnormal overlap relationship of the particle is detected, thereby breaking up the low-aggregation clustering assignment. This method effectively alleviates the over-aggregation phenomenon in particle clustering.

[0084] (4) Considering that the traditional InfoNCE loss function cannot guarantee a uniform distribution of similarity between negative sample pairs after training, this invention adds a similarity regularization term after the InfoNCE loss function to make the similarity between the anchor sample and each negative sample as small as possible, thereby achieving the expected training effect. Attached Figure Description

[0085] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0086] Figure 1 This is an overall flowchart of the method of the present invention.

[0087] Figure 2 This is a data processing framework diagram of the method of the present invention.

[0088] Figure 3 This is a flowchart of the algorithm of the method of the present invention. Detailed Implementation

[0089] like Figure 1 , Figure 2 and Figure 3 As shown, this embodiment of the invention provides a deep multi-granularity granulocyte-spherocyte contrastive clustering method for melanoma diagnosis.

[0090] Includes the following steps:

[0091] Step 1: Preprocess the skin disease image dataset to generate two sample image datasets;

[0092] Step 2: Use a pre-trained deep neural network to extract features from the two sample image datasets to obtain two sample feature sets.

[0093] Step 3: Use the multi-granularity particle-sphere comparison method for training. Construct negative samples by using particle-sphere clustering to extract features from the previous granularity layer, perform comparative learning, and use the extracted features for the next granularity layer. Repeat this process for training.

[0094] Step 4: Integrate the features extracted from each stage by concatenating the features from each stage of training element by element.

[0095] Step 5: Perform comparative clustering using an autoencoder. Add the two membership matrices obtained from the spliced ​​feature inputs to obtain the final clustering result.

[0096] Step 1 includes the following steps:

[0097] Step 1-1: After denoising and normalizing the original skin disease image dataset, the images are augmented by random flipping, color transformation and adding Gaussian noise of different intensities. This process is repeated twice to generate two augmented sample image datasets. The two sample image datasets are then merged into one dataset, denoted as D.

[0098] Steps 1-2, taking the ISIC2019 dataset as an example, the ISIC 2019 challenge is an open-source project focused on dermatopathology image classification. It aims to classify dermatopathology images using machine learning technology to help doctors diagnose skin diseases more accurately. The ISIC2019 dataset can be used to classify skin disease images in 9 different diagnostic categories. After merging the training and test sets, it contains 25,331 images. The images are preprocessed and scaled to 512×512 pixels, and then normalized with a mean of 0 and a variance of 1. Two sample image datasets are generated through data augmentation. The two sample image datasets are then merged into one dataset, denoted as D.

[0099] Step 2 includes the following steps:

[0100] Step 2-1: Use a pre-trained deep neural network to extract features from dataset D, capture high-level features of the input data, and obtain a sample feature set. Pre-trained deep neural networks are usually trained on large datasets and have strong generalization and transfer learning capabilities.

[0101] Step 2-2: Select ResNet34 as the pre-trained deep neural network. ResNet34 is a variant of a lightweight deep residual network that contains 34 layers. This network is designed to solve image recognition tasks. Two augmented sample image datasets are input into ResNet34. The input images need to be randomly cropped to 224×224 pixels.

[0102] Step 3 includes the following steps:

[0103] Step 3-1: The features of the sample feature set obtained in Step 2 are used as the features of the first granularity layer. Particle-sphere clustering is then performed on dataset D for negative sample sampling. A particle-sphere is a spherical structure covering a local sample, defined as a sphere with center c and radius r, used to represent a local region of the dataset. The calculation formula is:

[0104]

[0105] Where, x n It is the nth sample in the sphere, N b is the number of samples covered by the sphere, and ||.|| is the Euclidean norm;

[0106] Let the total number of samples in dataset D be N. First, construct a sphere from the entire dataset. Divide the sphere into two subspheres. Then, continue dividing each subsphere until the resulting sphere no longer meets the partitioning criteria. The final set of spheres is GB = {Ω1,Ω2,...,Ω} t}, where t represents the total number of particles in the particle set GB, Ω t This represents the t-th ball;

[0107] Let the sphere Ω covering the sample set be Y. Y The conditions for partitioning are met, and the particles are divided into two sub-spheres. and The specific division rules are as follows:

[0108]

[0109] Y α ={y i ∈Y,||y i -y α ||≤||y i -y β ||}(4),

[0110] Y β =YY α (5),

[0111] Where y i and y j Given any two samples in the sample set Y, use argmax(.) to find the two samples y that are farthest apart in the sample set Y. α and y β ;Y α Represents sub-spheres distributed by distance Covered sample set; Y β Represents sub-spheres distributed by distance The covered sample set;

[0112] Step 3-2: Based on the divided spheres, generate cluster assignments and construct negative sample pairs for the current granularity layer; let Ω be any two spheres in the sphere set GB. i and Ω j The centers of the spheres are c and c respectively. i and c j The radii are r i and r j If the following conditions are met:

[0113] ||c i -c j || <r i +r j +max(r i ,r j (6),

[0114] Then determine the two spheres Ω i and Ω j Overlapping, according to the conditions of formula (6), the particle clustering results are initially generated, and the particles with overlapping relationships are assigned to the same cluster. Since the overlap condition is set too broadly, it will lead to the clustering results being overly concentrated. Therefore, the abnormally assigned particles are detected by combining particle density and particle connection angle, and the clusters where the particles are located are further divided.

[0115] Let Ω be the particle. i Overlapping spheres are E i For OL i The number of particles in the middle, OL i Middle E i Individual spheres; Ω i The category is Q i For CL i The number of particles in the middle, OL i Q in the middle i Individual spheres; sphere Ω i The number of samples covered is n i , will OL i Any two spheres in the array are denoted as Ω. (i,ε) and Ω (i,ξ) ε≠ξ and 1≤ε≤E i , 1≤ξ≤E i , granules Ω i minimum overlap angle θ i Defined as:

[0116]

[0117] Among them, V ε Ω represents a granular particlei The center of the sphere points to the sphere Ω (i,ε) The vector of the center of the sphere, V ξ Ω represents a granular particle i The center of the sphere points to the sphere Ω (i,ξ) The vector of the center of the sphere; sphere Ω i density ρ i Defined as:

[0118]

[0119] If the granule Ω i The following conditions must be met:

[0120] and

[0121] Then determine the particle Ω i Cluster CL i Allocation error, CL i Further partitioning is needed; let θ i ε and ξ at the minimum value are respectively and Corresponding to OL i The granules in the middle are and CL i All the particles according to and The distance is further divided into 2 clusters. and Let CL i Any sphere in Ω [i,δ] , 1≤δ≤Q i The specific division rules are as follows:

[0122]

[0123] Where c δ , and Ω, respectively [i,δ] The center of the ball, The center of the ball and The center of the ball;

[0124] After performing step 3-2 detection on all spheres in the sphere set GB and reclassifying them into their corresponding clusters, assign the same label to samples covered by all spheres in the same cluster, setting the current label to the g-th sphere. c Grain size layer, g c ≥2 (excluding the first granularity layer, which is unlabeled), if any sample pair is in the second granularity layer to the gth granularity layer c If all granularity layers are assigned the same label, they constitute negative sample pairs for the current granularity layer;

[0125] Step 3-3: Based on the negative sample pairs of the second granularity layer, a contrastive learning method is used for the first stage of learning to obtain the feature representation of the second granularity layer; a multilayer perceptron (MLP) is used for feature mapping to maximize the similarity of negative sample pairs and minimize the similarity of positive sample pairs. The InfoNCE loss function of the i-th sample in dataset D is... Defined as:

[0126]

[0127] Where exp is the natural exponential function, z i It is the feature representation of the i-th sample after MLP mapping. It is z i The corresponding positive samples are represented by features after MLP mapping, where sim is the similarity function and τ is the similarity function. ins It is a temperature coefficient used to control the smoothness of the probability distribution, N neg It is the number of negative samples of the i-th sample, and the positive sample pair is the corresponding sample of two sample image datasets in dataset D;

[0128] The formula for calculating the similarity sim(z1,z2) between any two features z1 and z2 is:

[0129]

[0130] in Indicates transpose;

[0131] Add a regularization term to the InfoNCE loss function to control the similarity of negative sample pairs. The formula is:

[0132]

[0133] Complete objective function L ins Represented as:

[0134]

[0135] Where α is the balance coefficient of the regularization term;

[0136] Steps 3-4 are repeated from 3-1 to 3-3 for the second stage of training, yielding the feature representation of the third granularity layer. This process is repeated for a total of S stages. The final stage yields the feature representation of the (S+1)th granularity layer, where S satisfies:

[0137]

[0138] Among them, K sB is the number of clusters in the s-th stage of particle-sphere clustering, and B is the batch size.

[0139] Step 4 includes the following steps:

[0140] Step 4-1: Perform element-wise concatenation of the extracted features at each granularity level. Let the two enhanced sample image datasets obtained in Step 1 be sample set a and sample set b, respectively. The features of all samples at the s-th granularity level of sample set v are... The final sample set v contains all the concatenated sample features h v for:

[0141]

[0142] Step 5 includes the following steps:

[0143] Step 5-1: Input the features extracted from the two sample image datasets into the autoencoder. The encoding and decoding layers of the autoencoder output the encoded and reconstructed feature data respectively. The intermediate layer is the embedding layer of the autoencoder. The dimension is set to the final number of clusters. The output of the embedding layer is converted into the membership degree of each sample to each cluster, that is, the probability of each sample belonging to each cluster. At the same time, the network structure of the autoencoder is optimized by minimizing the mean square error between the input and output of the features.

[0144] Step 5-2, the definition of an autoencoder is:

[0145] h = f(W1x + b1)(18),

[0146] y = g(W2h + b2) (19),

[0147] Where x is the input sample feature data, W1, W2 and b1, b2 are coefficient matrices, f(·) is the encoder of the autoencoder, h is the output of the encoder, g(·) is the decoder of the autoencoder, and y is the output of the decoder. Both the encoder and decoder are fully connected layers with two or more layers, and the encoder and decoder have symmetrical structures.

[0148] Step 5-3: Optimize the autoencoder parameters by minimizing the decoder's reconstruction error. The autoencoder's reconstruction error L... re Using the minimum mean square error, it is defined as:

[0149]

[0150] Where, q i The features are concatenated from the i-th sample in D after S stages of learning. It is q i The output reconstructed by the decoder;

[0151] Step 5-4: Let the dimension of the autoencoder embedding layer be the number of clusters C. The output of the embedding layer represents the membership degree of each sample to each cluster. After normalizing the output, any element in the membership degree matrix of the sample set v is... satisfy:

[0152]

[0153] Step 5-5: Perform comparative training on the two membership matrices obtained by concatenating the features of the two sample image datasets in dataset D. Let the membership vector of the j-th sample in sample set a belong to each cluster be... The membership vector of the j-th sample in sample set b to each cluster is: The loss function of the j-th sample in sample set a The loss function of the j-th sample in sample set b They are respectively:

[0154]

[0155] Where, τ clu It is a temperature coefficient that controls the smoothness of the distribution; the clustering results are compared with the training objective function L. clu Represented as:

[0156]

[0157] Where H(Y) is the entropy of the probability distribution, expressed as:

[0158]

[0159] in, The average probability assigned to the k-th cluster in the sample set v is expressed as:

[0160]

[0161] Steps 5-6: The objective function L for autoencoder training ae Represented as:

[0162] L ae =L re +L clu (27);

[0163] Steps 5-7: After training, the two membership matrices are added together to form the final assignment matrix μ, where any element μ... jk Let represent the final probability of the j-th sample being assigned to the k-th cluster. The final clustering result is obtained based on the assignment matrix. That is, for each sample, the cluster with the highest value in the assignment matrix is ​​selected as the final cluster, as expressed by the following formula:

[0164]

[0165] Where Cluster(j) represents the clustering result of the j-th sample. Nine clusters were selected, and the final membership matrix is ​​shown in Table 1.

[0166] Table 1

[0167] <![CDATA[u i1 ]]> <![CDATA[u i2 ]]> … <![CDATA[u i8 ]]> <![CDATA[u i9 ]]> 0 0 0 … 0 0 1 0.51 0.06 … 0.01 0 … … … … … … 12663 0 0.12 … 0 0.05 12664 0.44 0.02 … 0.04 0.03 12665 0 0.61 … 0.03 0 12666 0 0.49 … 0 0 … … … … … … 25329 0 0 … 0.02 0 25330 0 0 … 0.01 0.02

[0168] As can be seen from the membership matrix results in Table 1, this invention can fully extract the feature information of skin disease images and produce diagnostic results with high confidence. Based on the clustering results, TP, FP, FN, and TN are calculated. TP represents the number of samples predicted as melanoma with a ground truth label of melanoma; FP represents the number of samples predicted as melanoma with a ground truth label of not being melanoma; FN represents the number of samples predicted as not melanoma with a ground truth label of melanoma; and TN represents the number of samples predicted as not melanoma with a ground truth label of not being melanoma. The final melanoma prediction accuracy (ACC) is calculated using the following method:

[0169]

[0170] The calculated ACC is 76.4%, which is higher than some current advanced models, indicating that the diagnostic results provided by this invention have high accuracy.

[0171] This invention provides a deep multi-granularity granulocyte-spherocyte contrastive clustering method for melanoma diagnosis. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A deep multi-granularity granule contrast clustering method for melanoma diagnosis, characterized in that, The method comprises the following steps: Step 1, preprocessing the dermatosis image dataset to generate two sample image datasets; Step 2, using a pre-trained deep neural network to extract features from the two sample image datasets to obtain two sample feature sets; Step 3, training using a multi-granularity particle ball comparison method, using particle ball clustering to construct negative samples from the features extracted in the previous granularity layer, performing contrastive learning, and using the extracted features for the next granularity layer, and sequentially training; Step 4, fusing the features extracted in each stage, and element-level splicing the features in each stage of each training; Step 5, using a self-encoder for contrastive clustering, adding the two membership matrices obtained by inputting the spliced features to obtain the final clustering result; Step 3 comprises the following steps: Step 3-1, the features of the sample feature set obtained in step 2 are taken as the first granularity layer features, and the data set is clustered into granules , and the granules are used for negative sample sampling; the granule is a spherical structure covering a local sample, defined as a sphere with a sphere center and a radius , used to represent a local area of the data set, and the calculation formula is: （1）， （2）， wherein, is the number of samples in the granulocyte, is the number of samples covered by the granulocyte, is the number of samples covered by the granulocyte, is the Euclidean norm; Set data set The total number of samples is , first, a particle ball is formed with the entire data set, the particle ball is divided into two sub-balls, and the two sub-balls are further divided until the divided particle ball does not meet the division condition , and finally, the particle ball set is obtained , The total number of particle balls in the particle ball set is represented by , and the tth particle ball is represented by Set the coverage sample set as The granule ball , meet the division condition, divided into two sub granule ball And , the specific division rule is: （3）， （4）， （5）， wherein and are any two samples in the sample set find the two samples in the sample set that are farthest apart and ; ; denotes the sample set covered by the sub-sphere assigned by distance ; denotes the sample set covered by the sub-sphere assigned by distance ; Step 3-2, according to the divided granules, generate cluster assignment, build the negative sample pair of the current granularity layer; set the granule set Any two granules And The center of the sphere is And The radius is And If the following conditions are met: （6）， then the two granules are determined to overlap and The granule clustering result is initially generated according to the condition of formula (6), the granules with overlapping relationship are assigned to the same class cluster, the granules with abnormal assignment are detected through the joint detection of granule density and granule connection angle, and the class cluster where the granules are located is further divided. Let the sphere be Overlapping spheres are , for The number of particles in the middle, express The Middle Each ball; The category is , for The number of particles in the middle, express The Middle Individual spheres; spheres The number of samples covered is ,Will Any two spheres in the array are denoted as . and , and , granules minimum overlap angle Defined as: （7）， wherein denotes the center of the sphere of the particle denotes the vector pointing from the center of the sphere of the particle to the center of the sphere of the particle denotes the vector pointing from the center of the sphere of the particle to the center of the sphere of the particle ; the density of the particle is defined as: ​ （8）， If granulocytes satisfy the following conditions: （9）， Then the granules are determined The cluster where the granules are located Abnormal distribution, Continue to divide; let The minimum value is obtained And Respectively And Corresponding to the granules in Are And All granules in According to the distance to And Continue to divide into two clusters And Let any granule in Be , The specific division rule is: （10）， （11）， wherein , and are the center of the sphere of the granulocyte , the center of the sphere of the granulocyte and the center of the sphere of the granulocyte , respectively. The granules are collected After all the granules in the same cluster complete the detection of step 3-2 and complete the corresponding cluster re-partitioning, the same label is assigned to the samples covered by all the granules in the same cluster, and the current is set to the first Granularity layer, If any sample pair is assigned to the same label at the second granularity layer to the Granularity layer, the negative sample pair of the current granularity layer is formed. Step 3-3, using the contrastive learning method to conduct the first stage learning according to the negative sample pairs of the second granularity layer, to obtain the feature representation of the second granularity layer; using a multi-layer perception to do the mapping processing of the features, to maximize the similarity of the negative sample pairs and minimize the similarity of the positive sample pairs, the dataset The InfoNCE loss function of the i-th sample in the middle is defined as: （12）， wherein is the natural exponential function, is the MLP mapped feature representation of the th sample feature representation, is the corresponding positive sample MLP mapped feature representation, is the similarity function, is the temperature coefficient; is the number of negative samples for the i-th sample, and the pair of positive samples is the corresponding sample pair in the dataset ​​ Any two features And The similarity of The calculation formula is: （13）， wherein denotes the transpose; Adding a regularization term to control the similarity of negative samples after the InfoNCE loss function The formula is: （14）， Complete objective function is represented as: （15）， wherein is a balancing coefficient of the regularization term; Step 3-4, repeat step 3-1 to step 3-3 to perform the 2nd stage training to obtain the feature representation of the 3rd granularity layer, and sequentially cycle, a total of stages, and finally the feature representation of the th granularity layer is trained to obtain.

2. The method of claim 1, wherein, In step 1, the dermatosis image dataset is denoised and normalized, and then two data enhancement processes are performed to generate two enhanced sample image datasets; the two enhanced sample image datasets are combined into one dataset ; the data enhancement process includes random flipping, color transformation, and adding Gaussian noise of different intensities.

3. The method of claim 2, wherein, In step 2, the pre-trained deep neural network is used to extract features from the dataset to capture high-level features of the input data, resulting in a set of sample features.

4. The method of claim 3, wherein, In steps 3-4, satisfies: （16）， wherein, is the number of clusters of the first stage granulocyte cluster, is the batch size.

5. The method of claim 4, wherein, In step 4, element-level splicing is performed on each granularity layer feature extracted, and the two enhanced sample image data sets obtained in step 1 are set as sample set and sample set , all sample features of the first granularity layer of sample set are , and all spliced sample features of the final sample set are : （17）。 6. The method of claim 5, wherein, Step 5 comprises the following steps: Step 5-1, inputting the features extracted from the two sample image datasets into the self-encoder, respectively outputting the encoded and reconstructed feature data through the encoding layer and decoding layer of the self-encoder, the intermediate layer being the embedding layer of the self-encoder, the dimension being set as the final number of clusters, and the output of the embedding layer being converted into the membership of each sample to each cluster, and the network structure of the self-encoder being optimized by minimizing the mean square error between the input and output of the features; In step 5-2, the self-encoder is defined as: （18）， （19）， wherein, is input sample feature data, , and , is a coefficient matrix, is an encoder of the autoencoder, is an output of the encoder, is a decoder of the autoencoder, is an output of the decoder, the encoder and the decoder are fully connected layers with more than two layers, and the encoder and the decoder are symmetric in structure. Step 5 - 3, optimizing parameters of the autoencoder by minimizing reconstruction error of the decoder, reconstruction error of the autoencoder Adopting the least mean square error, defined as: （20）， in, yes The Middle Each sample was processed Features pieced together after each stage of learning yes The output reconstructed by the decoder; Step 5 - 4, let the autoencoder embedding layer dimension be the number of clusters The output of the embedding layer is represented as the membership of each sample to each cluster. After normalization, any element in the membership matrix of the sample set satisfies: ​ （21）； Step 5-5, the two membership matrices obtained by the splicing feature input of the two sample image data sets in the data set D are compared and trained, and it is assumed that the membership vector of the first sample in the sample set D belongs to each class cluster is , the membership vector of the first sample in the sample set D belongs to each class cluster is , the loss function of the first sample in the sample set D is , and the loss function of the first sample in the sample set D is . ​​​​​​​​ （22）， （23）， wherein, is a temperature coefficient that controls the degree of smoothing of the distribution, the clustering result is compared with the target function of training is expressed as: （24）， wherein is the entropy of the probability distribution, expressed as: （25）， wherein, is the sample set assigning to the th cluster, is represented as: （26）； Step 5-6, objective function for autoencoder training is represented as: （27）； Step 5-7, after training, the two membership matrices are added to form the final assignment matrix , assignment matrix any element in the matrix represents the final probability of the th sample being assigned to the th cluster, the final clustering result is obtained according to the assignment matrix, that is, for each sample, the cluster with the highest value in the assignment matrix is selected as the final cluster.

7. The method of claim 6, wherein, In step 5-7, the final clustering result is represented by the following formula: （28）， wherein, represents the clustering result of the th sample.

8. An electronic device, comprising: A processor and a memory, the memory storing program code, when the program code is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

9. A storage medium, characterized by A computer program or instructions are stored, when the computer program or instructions are run on a computer, the steps of the method according to any one of claims 1 to 7 are executed.

Images