Deep multi-granularity particle ball contrast clustering method for melanoma diagnosis

Abstract

The invention provides a deep multi-granularity pellet contrast clustering method for melanoma diagnosis, and the method comprises the steps: 1, carrying out the preprocessing of a skin disease image data set, and generating two sample image data sets; 2, performing feature extraction on the sample image by using a pre-trained deep neural network; step 3, training by using a multi-granularity particle-ball comparison method; 4, fusing the features extracted in each stage; and 5, performing image contrast clustering by using an auto-encoder. According to the scheme, features of skin disease image data are automatically extracted, key features are accurately captured, the existing deep clustering effect is improved, an important classification basis is provided for prediction of melanoma patients, and doctors are helped to take treatment and rehabilitation measures in advance.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing, and in particular relates to a deep multi-granularity sphere comparison clustering method for melanoma diagnosis. Background Art

[0002] Melanoma is a malignant tumor that originates from melanocytes in the skin or mucous membranes. It is highly invasive and metastatic, making it one of the most dangerous types of skin cancer. Its incidence is increasing worldwide, making early detection and timely treatment crucial for melanoma patients. However, due to the complex nature of skin lesions, symptoms of conditions such as basal cell carcinoma and actinic keratosis are very similar to those of melanoma, making melanoma diagnosis challenging. Traditional manual evaluation methods for skin disease image data are subject to limitations such as high subjectivity, inefficiency, and data bias when processing this high-dimensional data, resulting in insufficient diagnostic accuracy for melanoma patients. Automatically integrating and analyzing these complex features to accurately predict melanoma diagnoses and thus assist physicians in clinical decision-making remains a challenging task. With the rapid advancement of technology, deep learning, which can process complex and high-dimensional data without human intervention, has shown significant application value in melanoma diagnosis.

[0003] Deep learning automatically learns useful features from large amounts of data by building multi-layered neural network models, enabling it to accomplish complex tasks. Clustering is a method for automatically labeling data by dividing samples in a dataset into clusters, ensuring that samples within the same cluster are as similar as possible and samples from different clusters are as different as possible. With its integration with deep learning, deep clustering has become a more efficient clustering method. Contrastive learning is a widely used technique in deep clustering. It utilizes positive and negative samples to guide deep learning models. Its core goal is to maximize the similarity between positive data pairs while minimizing the similarity between negative pairs.

[0004] Zhang et al., in "Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning" (Computer Vision–ECCV 2024, pp. 35-52), pointed out that in contrastive learning, latent features are not fully exploited, leading to the neglect of potentially valuable features, a phenomenon known as feature suppression. They proposed a feature-aware negative sampling method that constructs negative sample pairs through K-means clustering. However, K-means clustering requires specifying the number of clusters, making it inflexible for extracting features from datasets with varying sample data distributions. In "Research on Efficient Fuzzy Clustering Method Based on Local Fuzzy Granular Balls" (arXiv preprint, arXiv:2303.03590, 2023), Xie et al. proposed a granular clustering algorithm that does not require a specific number of clusters and can adapt to the distribution characteristics of sample data. This algorithm merges granules based on overlapping relationships to form the final clustering result. However, this method lacks detailed recognition of local structure and often suffers from over-aggregation when constructing overlapping relationships between granules. Therefore, how to generate discriminative granular clustering results during the model learning process and fully explore the potential features of skin disease images is a difficult problem faced by existing technologies. Summary of the Invention

[0005] Purpose of the Invention: This invention addresses the shortcomings of existing technologies by providing a deep multi-granularity sphere comparison clustering method for melanoma diagnosis. This method extracts image features using a pretrained deep neural network, then cyclically mines sample features using a multi-granularity sphere comparison method. The proposed sphere clustering method adapts to the distribution characteristics of sample pairs without requiring additional hyperparameters, resolving the over-aggregation problem of existing methods. Finally, the autoencoder mapping is trained using the comparison clustering method to obtain the final clustering results. This invention will provide a new approach for deep feature extraction and image clustering of skin disease images, providing a more scientific basis for melanoma diagnosis.

[0006] The method of the present invention comprises the following steps:

[0007] Step 1: preprocess the skin disease image dataset to generate two sample image datasets;

[0008] Step 2: Use a pre-trained deep neural network to extract features from the two sample image datasets to obtain two sample feature sets;

[0009] Step 3: Use the multi-granularity sphere comparison method to train. Use sphere clustering to construct negative samples for the features extracted from the previous granularity layer and perform comparative learning. The extracted features are used for the next granularity layer, and the training is repeated in sequence.

[0010] Step 4: Fuse the features extracted at each stage and perform element-level splicing on the features of each stage of each training;

[0011] Step 5: Use the autoencoder to perform comparative clustering, and add the two membership matrices obtained by the spliced feature input to obtain the final clustering result.

[0012] In step 1, the skin disease image dataset is denoised and normalized, and then data augmentation is performed twice to generate two enhanced sample image datasets; the two enhanced sample image datasets are merged into one dataset D; the data augmentation processing includes random flipping, color transformation, and adding Gaussian noise of different intensities.

[0013] In step 2, a pre-trained deep neural network is used to extract features from the dataset D, capturing the high-level features of the input data and obtaining a sample feature set. The pre-trained deep neural network has usually been trained on large datasets and has strong generalization and transfer learning capabilities.

[0014] Step 3 includes the following steps:

[0015] In step 3-1, the features of the sample feature set obtained in step 2 are used as the first granularity layer features to perform granular sphere clustering on the dataset D for negative sample sampling. A granular sphere is a spherical structure covering a local sample, defined as a sphere with a center of c and a radius of r, which is used to represent a local area of the dataset. The calculation formula is:

[0016]

[0017] Among them, x n is the nth sample in the sphere, N b is the number of samples covered by the sphere, ||.|| is the Euclidean norm;

[0018] Assume that the total number of samples in the data set D is N. First, the entire data set is used to form a ball, which is then divided into two sub-balls. The two sub-balls are then divided again until the divided ball no longer meets the division condition. The final set of spheres is GB = {Ω1,Ω2,...,Ω t}, t represents the total number of spheres in the sphere set GB, Ω t represents the t-th ball;

[0019] Let the covering sample set be the sphere Ω of Y Y, satisfying the division conditions, divided into two sub-spheres and The specific division rules are:

[0020]

[0021] Y α ={y i ∈Y,||y i -y α ||≤||y i -y β ||} (4),

[0022] Y β =YY α (5),

[0023] where y i and y j are any two samples in the sample set Y, and use argmax(.) to find the two samples y with the farthest distance in the sample set Y. α and y β ; Y α Indicates the distribution of particles by distance Covered sample set; Y β Indicates the distribution of particles by distance Covered sample set;

[0024] Step 3-2, based on the divided spheres, generate cluster assignments and construct negative sample pairs for the current granularity layer; let any two spheres Ω in the sphere set GB be i and Ω j The centers of the spheres are c i and c j , the radii are r i and r j , if the following conditions are met:

[0025] ||c i -c j || <r i +r j +max(r i ,r j ) (6),

[0026] Then determine the two balls Ω i and Ω j Overlapping: Based on the conditions of formula (6), the clustering results of the spheres are initially generated, and the spheres with overlapping relationships are assigned to the same cluster. Since the overlapping condition is set relatively broadly, it will lead to excessive concentration of clustering results. Therefore, the sphere density and sphere connection angle are used to jointly detect the abnormal spheres, and the clusters where the spheres are located are further divided.

[0027] Assume that the particle Ω i The set of overlapping spheres is E i For OL i The number of spheres in Indicates OL i Middle E i spheres; Ω i The cluster is Q i CL i The number of spheres in Indicates OL i Q i sphere; sphere Ω i The number of samples covered is n i , will OL i Any two spheres in the (i,ε) and Ω (i,ξ) , ε≠ξ and 1≤ε≤E i , 1≤ξ≤E i , particle Ω i The minimum overlap angle θ i Defined as:

[0028]

[0029] Among them, V ε Represents the particle Ω i The center of the sphere points to the particle Ω (i,ε) The vector of the center of the sphere, V ξ Represents the particle Ω i The center of the sphere points to the particle Ω (i,ξ) The vector of the center of the sphere; the particle sphere Ω i The density ρ i Defined as:

[0030]

[0031] If the particle Ω i The following conditions are met:

[0032] and

[0033] Then determine the particle Ω i Cluster CL i Allocation exception, CL i Need to continue dividing; let θ i When the minimum value is achieved, ε and ξ are and Corresponding to OL i The sphere in and CLi All the balls in and The distance is further divided into two clusters and Assume CL i Any ball in the [i,δ] , 1≤δ≤Q i , the specific division rules are:

[0034]

[0035] where c δ , and Ω [i,δ] The center of the ball, The center of the sphere and The center of the sphere;

[0036] After all the balls in the ball set GB have completed the detection of step 3-2 and completed the corresponding cluster re-division, the same label is assigned to all samples covered by the balls in the same cluster, and the current one is set to the gth c Granularity layer, g c ≥2 (excluding the first granularity layer, which has no labels), if any sample pair is in the second granularity layer to the gth c If all granularity layers are assigned the same label, they constitute the negative sample pairs of the current granularity layer;

[0037] Step 3-3, based on the negative sample pairs of the second granularity layer, the contrastive learning method is used to perform the first stage learning to obtain the feature representation of the second granularity layer; the multi-layer perceptron (MLP) is used for feature mapping processing to maximize the similarity of negative sample pairs and minimize the similarity of positive sample pairs. The InfoNCE loss function of the i-th sample in the dataset D is Defined as:

[0038]

[0039] Where exp is the natural exponential function, z i is the feature representation of the i-th sample after MLP mapping, It is z i The corresponding positive sample is represented by the feature after MLP mapping, sim is the similarity function, τ ins is the temperature coefficient, which is used to control the smoothness of the probability distribution, N neg is the number of negative samples of the i-th sample, and the positive sample pair is the corresponding sample of the two sample image datasets in the dataset D;

[0040] The calculation formula for the similarity sim(z1,z2) between any two features z1 and z2 is:

[0041]

[0042] in represents transpose;

[0043] Add a regularization term to control the similarity of negative samples after the InfoNCE loss function The formula is:

[0044]

[0045] The complete objective function L ins Expressed as:

[0046]

[0047] Among them, α is the balance coefficient of the regularization term;

[0048] Step 3-4: Repeat steps 3-1 to 3-3 to perform the second stage of training to obtain the feature representation of the third granularity layer. Repeat this process for a total of S stages. The last stage of training obtains the feature representation of the S+1th granularity layer.

[0049] In steps 3-4, S satisfies:

[0050]

[0051] Among them, K s is the number of clusters of granular sphere clustering in the sth stage, and B is the batch size.

[0052] In step 4, the extracted features of each granularity layer are spliced at the element level. The two enhanced sample image datasets obtained in step 1 are set as sample set a and sample set b. The features of all samples in the sth granularity layer of sample set v are All concatenated sample features h of the final sample set v v for:

[0053]

[0054] Step 5 includes the following steps:

[0055] Step 5-1: Input the features extracted from the two sample image datasets into the autoencoder respectively. The encoding layer and decoding layer of the autoencoder respectively output the encoded and reconstructed feature data. The middle layer is the embedding layer of the autoencoder. The dimension is set to the final number of clusters. The output of the embedding layer is converted into the membership degree of each sample corresponding to each cluster, that is, the probability of each sample belonging to each cluster. At the same time, the network structure of the autoencoder is optimized by minimizing the mean square error between the input and output of the features.

[0056] In step 5-2, the definition of the autoencoder is:

[0057] h=f(W1x+b1)(18),

[0058] y=g(W2h+b2)(19),

[0059] Where x is the input sample feature data, W1, W2 and b1, b2 are coefficient matrices, f(·) is the encoder of the autoencoder, h is the output of the encoder, g(·) is the decoder of the autoencoder, and y is the output of the decoder. Both the encoder and decoder have more than two fully connected layers, and the encoder and decoder structures are symmetrical.

[0060] Step 5-3, by minimizing the reconstruction error of the decoder, optimize the parameters of the autoencoder, the reconstruction error of the autoencoder L re The minimum mean square error is used, which is defined as:

[0061]

[0062] Among them, q i is the concatenated feature of the i-th sample in D after S stages of learning, It is q i The output reconstructed by the decoder;

[0063] In step 5-4, let the dimension of the autoencoder embedding layer be the number of clusters C. The output of the embedding layer is represented by the membership of each sample to each cluster. After normalizing the output, any element in the membership matrix of the sample set v is satisfy:

[0064]

[0065] Step 5-5, perform comparative training on the two membership matrices obtained by inputting the spliced features of the two sample image datasets in dataset D. Suppose the membership vector of the jth sample in sample set a belongs to each cluster is The membership vector of the jth sample in the sample set b belongs to each cluster is The loss function of the jth sample in sample set a And the loss function of the jth sample in sample set b They are:

[0066]

[0067] Among them, τ clu It is the temperature coefficient that controls the degree of distribution smoothness. The clustering results are compared with the objective function L of the training. clu Expressed as:

[0068]

[0069] Among them, H(Y) is the entropy of the probability distribution, which is expressed as:

[0070]

[0071] in, is the average probability assigned to the kth cluster in the sample set v, expressed as:

[0072]

[0073] Step 5-6, the objective function L of the autoencoder training ae Expressed as:

[0074] L ae =L re +L clu (27);

[0075] Steps 5-7, after the training is completed, the two membership matrices are added together to form the final allocation matrix μ, and any element μ in the allocation matrix μ is jk It represents the final probability of assigning the jth sample to the kth cluster. The final clustering result is obtained according to the assignment matrix. That is, for each sample, the cluster with the highest value in the assignment matrix is selected as the final cluster.

[0076] In steps 5-7, the final clustering result is expressed by the following formula:

[0077]

[0078] Among them, Cluster(j) represents the clustering result of the j-th sample.

[0079] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the method.

[0080] The present invention also provides a storage medium storing a computer program or instruction, which executes the steps of the method when the computer program or instruction is run on a computer.

[0081] The present invention has the following beneficial effects: (1) The present invention uses a multi-granularity contrastive learning and fusion method, which can fully mine the potential features in the sample set and provide richer and more distinguishable feature representations for clustering tasks.

[0082] (2) Based on the idea of granular clustering, the present invention uses a granular structure to construct negative sample pairs for contrastive learning. Compared with the K-means method, it does not require additional hyperparameters and can adapt to the distribution characteristics of samples, thereby improving the accuracy of negative sampling.

[0083] (3) The present invention redesigns a parameter-free sphere clustering method. First, the initial overlap constraint is relaxed to generate preliminary cluster assignments. Then, based on the internal density of the spheres and the sphere overlap angle, spheres with abnormal overlapping relationships are detected, thereby breaking up cluster assignments with low aggregation. This method effectively alleviates the over-aggregation phenomenon in sphere clustering.

[0084] (4) Considering that the traditional InfoNCE loss function cannot ensure the uniform distribution of similarity between negative samples after training, the present invention adds a similarity regularization term after the InfoNCE loss function to make the similarity between the anchor sample and each negative sample as small as possible, thereby achieving the expected training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.

[0086] Figure 1 1 is an overall flow chart of the method of the present invention.

[0087] Figure 2 This is a data processing framework diagram of the method of the present invention.

[0088] Figure 3 Flowchart of the algorithm of the method of the present invention. DETAILED DESCRIPTION

[0089] like Figure 1 、 Figure 2 and Figure 3 As shown, the embodiment of the present invention provides a deep multi-granularity sphere comparison clustering method for melanoma diagnosis.

[0090] The steps include:

[0091] Step 1: preprocess the skin disease image dataset to generate two sample image datasets;

[0092] Step 2: Use a pre-trained deep neural network to extract features from the two sample image datasets to obtain two sample feature sets;

[0093] Step 3: Use the multi-granularity sphere comparison method to train. Use sphere clustering to construct negative samples for the features extracted from the previous granularity layer and perform comparative learning. The extracted features are used for the next granularity layer, and the training is repeated in sequence.

[0094] Step 4: Fuse the features extracted at each stage and perform element-level splicing on the features of each stage of each training;

[0095] Step 5: Use the autoencoder to perform comparative clustering, and add the two membership matrices obtained by the spliced feature input to obtain the final clustering result.

[0096] Step 1 includes the following steps:

[0097] In step 1-1, after denoising and normalizing the original skin disease image dataset, data augmentation is performed on the image by random flipping, color conversion, and adding Gaussian noise of different intensities. This is repeated twice to generate two enhanced sample image datasets. The two sample image datasets are then merged into one dataset, denoted as D.

[0098] In steps 1-2, we use the ISIC2019 dataset as an example. The ISIC 2019 Challenge is an open-source project focused on skin pathology image classification. It aims to classify skin pathology images using machine learning techniques to help doctors more accurately diagnose skin diseases. The ISIC2019 dataset can be used to classify skin disease images in nine different diagnostic categories. The training and test sets are combined to contain 25,331 images. These images are pre-processed and resized to 512 × 512 pixels. They are then normalized with a mean of 0 and a variance of 1. Data augmentation is used to generate two sample image datasets, which are then merged into a single dataset, denoted as D.

[0099] Step 2 includes the following steps:

[0100] In step 2-1, a pre-trained deep neural network is used to extract features from the dataset D, capturing high-level features of the input data and obtaining a sample feature set. The pre-trained deep neural network has usually been trained on large datasets and has strong generalization and transfer learning capabilities.

[0101] In step 2-2, select ResNet34 as the pre-trained deep neural network. ResNet34 is a variant of a lightweight deep residual network with 34 layers. This network is designed for image recognition tasks. Input two enhanced sample image datasets into ResNet34. The input images need to be randomly cropped to 224×224 pixels.

[0102] Step 3 includes the following steps:

[0103] In step 3-1, the features of the sample feature set obtained in step 2 are used as the first granularity layer features to perform granular sphere clustering on the dataset D for negative sample sampling. A granular sphere is a spherical structure covering a local sample, defined as a sphere with a center of c and a radius of r, which is used to represent a local area of the dataset. The calculation formula is:

[0104]

[0105] Among them, x n is the nth sample in the sphere, N b is the number of samples covered by the sphere, ||.|| is the Euclidean norm;

[0106] Assume that the total number of samples in the data set D is N. First, the entire data set is used to form a ball, which is then divided into two sub-balls. The two sub-balls are then divided again until the divided ball no longer meets the division condition. The final set of spheres is GB = {Ω1,Ω2,...,Ω t}, t represents the total number of spheres in the sphere set GB, Ω t represents the t-th ball;

[0107] Let the covering sample set be the sphere Ω of Y Y , satisfying the division conditions, divided into two sub-spheres and The specific division rules are:

[0108]

[0109] Y α ={y i ∈Y,||y i -y α ||≤||y i -y β ||}(4),

[0110] Y β =YY α (5),

[0111] where y i and y j are any two samples in the sample set Y, and use argmax(.) to find the two samples y with the farthest distance in the sample set Y. α and y β ; Y α Indicates the distribution of particles by distance Covered sample set; Y β Indicates the distribution of particles by distance Covered sample set;

[0112] Step 3-2, based on the divided spheres, generate cluster assignments and construct negative sample pairs for the current granularity layer; let any two spheres Ω in the sphere set GB be i and Ω j The centers of the spheres are c i and c j , the radii are r i and r j , if the following conditions are met:

[0113] ||c i -c j || <r i +r j +max(r i ,r j ) (6),

[0114] Then determine the two balls Ω i and Ω j Overlapping: Based on the conditions of formula (6), the clustering results of the spheres are initially generated, and the spheres with overlapping relationships are assigned to the same cluster. Since the overlapping condition is set relatively broadly, it will lead to excessive concentration of clustering results. Therefore, the sphere density and sphere connection angle are used to jointly detect the abnormal spheres, and the clusters where the spheres are located are further divided.

[0115] Assume that the particle Ω i The set of overlapping spheres is E i For OL i The number of spheres in Indicates OL i Middle E i spheres; Ω i The cluster is Q i CL i The number of spheres in Indicates OL i Q i sphere; sphere Ω i The number of samples covered is n i , will OL i Any two spheres in the (i,ε) and Ω (i,ξ) , ε≠ξ and 1≤ε≤E i , 1≤ξ≤E i , particle Ω i The minimum overlap angle θ i Defined as:

[0116]

[0117] Among them, V ε Represents the particle Ωi The center of the sphere points to the particle Ω (i,ε) The vector of the center of the sphere, V ξ Represents the particle Ω i The center of the sphere points to the particle Ω (i,ξ) The vector of the center of the sphere; the particle sphere Ω i The density ρ i Defined as:

[0118]

[0119] If the particle Ω i The following conditions are met:

[0120] and

[0121] Then determine the particle Ω i Cluster CL i Allocation exception, CL i Need to continue dividing; let θ i When the minimum value is achieved, ε and ξ are and Corresponding to OL i The sphere in and CL i All the balls in and The distance is further divided into two clusters and Assume CL i Any ball in the [i,δ] , 1≤δ≤Q i , the specific division rules are:

[0122]

[0123] where c δ , and Ω [i,δ] The center of the ball, The center of the sphere and The center of the sphere;

[0124] After all the balls in the ball set GB have completed the detection of step 3-2 and completed the corresponding cluster re-division, the same label is assigned to all samples covered by the balls in the same cluster, and the current one is set to the gth c Granularity layer, g c ≥2 (excluding the first granularity layer, which has no labels), if any sample pair is in the second granularity layer to the gth c If all granularity layers are assigned the same label, they constitute the negative sample pairs of the current granularity layer;

[0125] Step 3-3, based on the negative sample pairs of the second granularity layer, the contrastive learning method is used to perform the first stage learning to obtain the feature representation of the second granularity layer; the multi-layer perceptron (MLP) is used for feature mapping processing to maximize the similarity of negative sample pairs and minimize the similarity of positive sample pairs. The InfoNCE loss function of the i-th sample in the dataset D is Defined as:

[0126]

[0127] Where exp is the natural exponential function, z i is the feature representation of the i-th sample after MLP mapping, It is z i The corresponding positive sample is represented by the feature after MLP mapping, sim is the similarity function, τ ins is the temperature coefficient, which is used to control the smoothness of the probability distribution, N neg is the number of negative samples of the i-th sample, and the positive sample pair is the corresponding sample of the two sample image datasets in the dataset D;

[0128] The calculation formula for the similarity sim(z1,z2) between any two features z1 and z2 is:

[0129]

[0130] in represents transpose;

[0131] Add a regularization term to control the similarity of negative samples after the InfoNCE loss function The formula is:

[0132]

[0133] The complete objective function L ins Expressed as:

[0134]

[0135] Among them, α is the balance coefficient of the regularization term;

[0136] Step 3-4: Repeat steps 3-1 to 3-3 to perform the second stage of training to obtain the feature representation of the third granularity layer. Repeat this process for a total of S stages. The last stage of training obtains the feature representation of the S+1th granularity layer. S satisfies:

[0137]

[0138] Among them, K sis the number of clusters of granular sphere clustering in the sth stage, and B is the batch size;

[0139] The step 4 comprises the following steps:

[0140] Step 4-1, perform element-level splicing on the extracted features of each granularity layer, set the two enhanced sample image datasets obtained in step 1 as sample set a and sample set b, and all sample features of the sth granularity layer of sample set v are All concatenated sample features h of the final sample set v v for:

[0141]

[0142] The step 5 comprises the following steps:

[0143] Step 5-1: Input the features extracted from the two sample image datasets into the autoencoder respectively. The encoding layer and decoding layer of the autoencoder respectively output the encoded and reconstructed feature data. The middle layer is the embedding layer of the autoencoder. The dimension is set to the final number of clusters. The output of the embedding layer is converted into the membership degree of each sample corresponding to each cluster, that is, the probability of each sample belonging to each cluster. At the same time, the network structure of the autoencoder is optimized by minimizing the mean square error between the input and output of the features.

[0144] In step 5-2, the definition of the autoencoder is:

[0145] h=f(W1x+b1)(18),

[0146] y=g(W2h+b2)(19),

[0147] Where x is the input sample feature data, W1, W2 and b1, b2 are coefficient matrices, f(·) is the encoder of the autoencoder, h is the output of the encoder, g(·) is the decoder of the autoencoder, and y is the output of the decoder. Both the encoder and decoder have more than two fully connected layers, and the encoder and decoder structures are symmetrical.

[0148] Step 5-3, by minimizing the reconstruction error of the decoder, optimize the parameters of the autoencoder, the reconstruction error of the autoencoder L re The minimum mean square error is used, which is defined as:

[0149]

[0150] Among them, q i is the concatenated feature of the i-th sample in D after S stages of learning, It is q i The output reconstructed by the decoder;

[0151] In step 5-4, let the dimension of the autoencoder embedding layer be the number of clusters C. The output of the embedding layer is represented by the membership of each sample to each cluster. After normalizing the output, any element in the membership matrix of the sample set v is satisfy:

[0152]

[0153] Step 5-5, perform comparative training on the two membership matrices obtained by inputting the spliced features of the two sample image datasets in dataset D. Suppose the membership vector of the jth sample in sample set a belongs to each cluster is The membership vector of the jth sample in the sample set b belongs to each cluster is The loss function of the jth sample in sample set a And the loss function of the jth sample in sample set b They are:

[0154]

[0155] Among them, τ clu It is the temperature coefficient that controls the degree of distribution smoothness. The clustering results are compared with the objective function L of the training. clu Expressed as:

[0156]

[0157] Among them, H(Y) is the entropy of the probability distribution, which is expressed as:

[0158]

[0159] in, is the average probability assigned to the kth cluster in the sample set v, expressed as:

[0160]

[0161] Step 5-6, the objective function L of the autoencoder training ae Expressed as:

[0162] L ae =L re +L clu (27);

[0163] Steps 5-7: After the training is completed, the two membership matrices are added together to form the final distribution matrix μ, where any element μ jk It represents the final probability of assigning the jth sample to the kth cluster. The final clustering result is obtained according to the assignment matrix. That is, for each sample, the cluster with the highest value in the assignment matrix is selected as the final cluster, which is expressed by the following formula:

[0164]

[0165] Cluster(j) represents the clustering result of the jth sample. The number of clusters selected is 9, and the final membership matrix is shown in Table 1:

[0166] Table 1

[0167] <![CDATA[u i1 ]]> <![CDATA[u i2 ]]> … <![CDATA[u i8 ]]> <![CDATA[u i9 ]]> 0 0 0 … 0 0 1 0.51 0.06 … 0.01 0 … … … … … … 12663 0 0.12 … 0 0.05 12664 0.44 0.02 … 0.04 0.03 12665 0 0.61 … 0.03 0 12666 0 0.49 … 0 0 … … … … … … 25329 0 0 … 0.02 0 25330 0 0 … 0.01 0.02

[0168] As can be seen from the membership matrix results in Table 1, the present invention can fully extract the characteristic information of skin disease images and produce diagnostic results with high confidence. Based on the clustering results, TP, FP, FN, and TN are calculated. TP is the number of samples predicted to be melanoma and whose true value label is also melanoma, FP is the number of samples predicted to be melanoma and whose true value label is not melanoma, FN is the number of samples predicted not to be melanoma and whose true value label is melanoma, and TN is the number of samples predicted not to be melanoma and whose true value label is not melanoma. The final melanoma prediction accuracy ACC is calculated using the following method:

[0169]

[0170] The calculated ACC is 76.4%, which is ahead of some current advanced models. It can be seen that the diagnostic results provided by the present invention are highly accurate.

[0171] The present invention provides a deep multi-granularity sphere comparative clustering method for melanoma diagnosis. There are numerous methods and approaches for implementing this technical solution. The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. A deep multi-granularity sphere comparison clustering method for melanoma diagnosis, characterized in that: The following steps are involved: Step 1: preprocess the skin disease image dataset to generate two sample image datasets; Step 2: Use a pre-trained deep neural network to extract features from the two sample image datasets to obtain two sample feature sets; Step 3: Use the multi-granularity sphere comparison method to train. Use sphere clustering to construct negative samples for the features extracted from the previous granularity layer and perform comparative learning. The extracted features are used for the next granularity layer, and the training is repeated in sequence. Step 4: Fuse the features extracted at each stage and perform element-level splicing on the features of each stage of each training; Step 5: Use the autoencoder to perform comparative clustering, and add the two membership matrices obtained by the spliced feature input to obtain the final clustering result.

2. The method according to claim 1, characterized in that In step 1, the skin disease image dataset is denoised and normalized, and then data augmentation is performed twice to generate two enhanced sample image datasets; the two enhanced sample image datasets are merged into one dataset D; the data augmentation processing includes random flipping, color transformation, and adding Gaussian noise of different intensities.

3. The method according to claim 2, characterized in that In step 2, a pre-trained deep neural network is used to extract features from the dataset D, capturing high-level features of the input data and obtaining a sample feature set.

4. The method according to claim 3, characterized in that Step 3 includes the following steps: In step 3-1, the features of the sample feature set obtained in step 2 are used as the first granularity layer features to perform granular sphere clustering on the dataset D for negative sample sampling. A granular sphere is a spherical structure covering a local sample, defined as a sphere with a center of c and a radius of r, which is used to represent a local area of the dataset. The calculation formula is: Among them, x n is the nth sample in the sphere, N b is the number of samples covered by the sphere, ||.|| is the Euclidean norm; Assume that the total number of samples in the data set D is N. First, the entire data set is used to form a ball, which is then divided into two sub-balls. The two sub-balls are then divided again until the divided ball no longer meets the division condition. The final set of spheres is GB = {Ω1,Ω2,...,Ω t }, t represents the total number of spheres in the sphere set GB, Ω t represents the t-th ball; Let the covering sample set be the sphere Ω of Y Y , satisfying the division conditions, divided into two sub-spheres and The specific division rules are: AND α ={and i ∈Y,||y i -and α ||≤||and i -and β ||}(4), AND β =YY α (5), where y i and y j are any two samples in the sample set Y, and use argmax(.) to find the two samples y with the farthest distance in the sample set Y. α and y β ; Y α Indicates the distribution of particles by distance Covered sample set; Y β Indicates the distribution of particles by distance Covered sample set; Step 3-2, based on the divided spheres, generate cluster assignments and construct negative sample pairs for the current granularity layer; let any two spheres Ω in the sphere set GB be i and Ω j The centers of the spheres are c i and c j , the radii are r i and r j , if the following conditions are met: ||c i -c j ||<r i +r j +max(r i ,r j ) (6), Then determine the two balls Ω i and Ω j Overlapping: Based on the conditions of formula (6), the clustering results of the spheres are initially generated. The spheres with overlapping relationships are assigned to the same cluster. Then, the sphere density and the sphere connection angle are used to jointly detect the abnormally assigned spheres, and the clusters where the spheres are located are further divided. Assume that the particle Ω i The set of overlapping spheres is E i For OL i The number of spheres in Indicates OL i Middle E i spheres; Ω i The cluster is Q i CL i The number of spheres in Indicates OL i Q i sphere; sphere Ω i The number of samples covered is n i , will OL i Any two spheres in the (i,ε) and Ω (i,ξ) , ε≠ξ and 1≤ε≤E i , 1≤ξ≤E i , particle Ω i The minimum overlap angle θ i Defined as: Among them, V ε Represents the particle Ω i The center of the sphere points to the particle Ω (i,ε) The vector of the center of the sphere, V ξ Represents the particle Ω i The center of the sphere points to the particle Ω (i,ξ) The vector of the center of the sphere; the particle sphere Ω i The density ρ i Defined as: If the particle Ω i The following conditions are met: and Then determine the particle Ω i Cluster CL i Allocation exception, CL i Need to continue dividing; let θ i When the minimum value is achieved, ε and ξ are and Corresponding to OL i The sphere in and CL i All the balls in and The distance is further divided into two clusters and Assume CL i Any ball in the [i,δ] , 1≤δ≤Q i , the specific division rules are: where c δ , and Ω [i,δ] The center of the ball, The center of the sphere and The center of the sphere; After all the balls in the ball set GB have completed the detection of step 3-2 and completed the corresponding cluster re-division, the same label is assigned to all samples covered by the balls in the same cluster, and the current one is set to the g-th c Granularity layer, g c ≥2, if any sample pair is in the 2nd granularity layer to the gth c If all granularity layers are assigned the same label, they constitute the negative sample pairs of the current granularity layer; Step 3-3, based on the negative sample pairs of the second granularity layer, the contrastive learning method is used to perform the first stage learning to obtain the feature representation of the second granularity layer; the multi-layer perceptron is used to perform feature mapping processing to maximize the similarity of the negative sample pairs and minimize the similarity of the positive sample pairs. The InfoNCE loss function of the i-th sample in the dataset D is Defined as: Where exp is the natural exponential function, z i is the feature representation of the i-th sample after MLP mapping, It is z i The corresponding positive sample is represented by the feature after MLP mapping, sim is the similarity function, τ ins is the temperature coefficient; N neg is the number of negative samples of the i-th sample, and the positive sample pair is the corresponding sample of the two sample image datasets in the dataset D; The calculation formula for the similarity sim(z1,z2) between any two features z1 and z2 is: Where T represents transpose; Add a regularization term to control the similarity of negative samples after the InfoNCE loss function The formula is: The complete objective function L ins Expressed as: Among them, α is the balance coefficient of the regularization term; Step 3-4: Repeat steps 3-1 to 3-3 to perform the second stage of training to obtain the feature representation of the third granularity layer. Repeat this process for a total of S stages. The last stage of training obtains the feature representation of the S+1th granularity layer.

5. The method according to claim 4, characterized in that In steps 3-4, S satisfies: Among them, K s is the number of clusters of granular sphere clustering in the sth stage, and B is the batch size.

6. The method according to claim 5, characterized in that In step 4, the extracted features of each granularity layer are spliced at the element level. The two enhanced sample image datasets obtained in step 1 are set as sample set a and sample set b. The features of all samples in the sth granularity layer of sample set v are All concatenated sample features h of the final sample set v v for:

7. The method according to claim 6, characterized in that Step 5 includes the following steps: Step 5-1: Input the features extracted from the two sample image datasets into the autoencoder respectively. The encoding layer and decoding layer of the autoencoder respectively output the encoded and reconstructed feature data. The middle layer is the embedding layer of the autoencoder. The dimension is set to the final number of clusters. The output of the embedding layer is converted into the membership degree of each sample corresponding to each cluster. At the same time, the network structure of the autoencoder is optimized by minimizing the mean square error between the input and output of the features. In step 5-2, the definition of the autoencoder is: h=f(W1x+b1)(18), y=g(W2h+b2)(19), Where x is the input sample feature data, W1, W2 and b1, b2 are coefficient matrices, f(·) is the encoder of the autoencoder, h is the output of the encoder, g(·) is the decoder of the autoencoder, and y is the output of the decoder. Both the encoder and decoder have more than two fully connected layers, and the encoder and decoder structures are symmetrical. Step 5-3, by minimizing the reconstruction error of the decoder, optimize the parameters of the autoencoder, the reconstruction error of the autoencoder L re The minimum mean square error is used, which is defined as: Among them, q i is the concatenated feature of the i-th sample in D after S stages of learning, It's q i The output reconstructed by the decoder; In step 5-4, let the dimension of the autoencoder embedding layer be the number of clusters C. The output of the embedding layer is represented by the membership of each sample to each cluster. After normalizing the output, any element in the membership matrix of the sample set v is satisfy: Step 5-5, perform comparative training on the two membership matrices obtained by inputting the spliced features of the two sample image datasets in dataset D. Suppose the membership vector of the jth sample in sample set a belongs to each cluster is The membership vector of the jth sample in the sample set b belongs to each cluster is The loss function of the jth sample in sample set a And the loss function of the jth sample in sample set b They are: Among them, τ clu It is the temperature coefficient that controls the degree of distribution smoothness. The clustering results are compared with the objective function L of the training. clu Expressed as: Among them, H(Y) is the entropy of the probability distribution, which is expressed as: in, is the average probability assigned to the kth cluster in the sample set v, expressed as: Step 5-6, the objective function L of the autoencoder training ae Expressed as: L ae =L re +L clu (27); Steps 5-7, after the training is completed, the two membership matrices are added together to form the final allocation matrix μ, and any element μ in the allocation matrix μ is jk It represents the final probability of assigning the jth sample to the kth cluster. The final clustering result is obtained according to the assignment matrix. That is, for each sample, the cluster with the highest value in the assignment matrix is selected as the final cluster.

8. The method according to claim 7, characterized in that In steps 5-7, the final clustering result is expressed by the following formula: Among them, Cluster(j) represents the clustering result of the j-th sample.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: A computer program or instruction is stored, and when the computer program or instruction is run on a computer, the steps of the method according to any one of claims 1 to 8 are executed.

Images