Medical image classification method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a medical image classification method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the steps of performing model training according to a source medical image and a disease category label at a first medical data end to obtain a source medical image classification model, and performing model training according to a target medical image and the source medical image classification model at a second medical data end to obtain a target medical image classification model, and image classification is performed on the target medical image through the target medical image classification model to obtain the target disease category, so that the difficulty of medical image classification can be reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, is applicable to the field of digital medicine, and in particular to a medical image classification method and device, electronic equipment, and storage medium. Background Art

[0002] The training of medical image classification models requires a large number of medical image samples, but the data distribution and annotation of medical image samples vary between different medical institutions. For example, the shapes and brightness of tissues and organs photographed by different medical institutions are different, and the imaging modes of medical image samples are different. In addition, labeling medical image samples consumes a lot of time, which makes medical image classification more difficult. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a medical image classification method and device, an electronic device and a storage medium, aiming to reduce the difficulty of medical image classification.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a medical image classification method, which is applied to a medical image classification device, wherein the medical image classification device includes a first medical data end and a second medical data end, wherein the first medical data end stores a source medical image and a disease category label of the source medical image, and the second medical data end stores a target medical image, wherein the method includes:

[0005] At the first medical data end, model training is performed according to the source medical image and the disease category label to obtain a source medical image classification model;

[0006] At the second medical data end, model training is performed based on the target medical image and the source medical image classification model to obtain a target medical image classification model, and the target medical image is classified by the target medical image classification model to obtain a target disease category.

[0007] In some embodiments, performing model training according to the target medical image and the source medical image classification model to obtain the target medical image classification model includes:

[0008] Performing mean calculation on the target medical image to obtain an image pixel mean;

[0009] Clustering the target medical image according to the image pixel mean to obtain a medical image cluster and a target cluster weight of the medical image cluster;

[0010] Training the source medical image classification model according to the medical image cluster to obtain a reference classification model;

[0011] The reference classification model is model aggregated according to the target cluster weight to obtain the target medical image classification model.

[0012] In some embodiments, clustering the target medical image according to the image pixel means to obtain medical image clusters and target cluster weights of the medical image clusters includes:

[0013] Calculating a distribution probability value that the image pixel mean belongs to a preset Gaussian distribution according to the image pixel mean, the preset cluster weight, the preset Gaussian mean and the preset Gaussian variance; the preset Gaussian distribution is determined by the preset Gaussian mean and the preset Gaussian variance;

[0014] Update the preset Gaussian mean according to the distribution probability value and the image pixel mean to obtain a reference Gaussian mean;

[0015] Update the preset Gaussian variance according to the image pixel mean, the distribution probability value and the reference Gaussian mean to obtain a reference Gaussian variance;

[0016] Update the preset cluster weight according to the distribution probability value to obtain a reference cluster weight;

[0017] Determining first target loss data according to the image pixel mean, the reference Gaussian mean, the reference Gaussian variance and the reference cluster weight;

[0018] Adjust the reference cluster weight according to the first target loss data to obtain the target cluster weight;

[0019] Adjust the reference Gaussian mean and the reference Gaussian variance according to the first target loss data to obtain a target Gaussian distribution;

[0020] The target medical image is divided into image clusters according to the target Gaussian distribution and the target cluster weights to obtain the medical image clusters.

[0021] In some embodiments, performing image segmentation on the target medical image according to the target Gaussian distribution and the target cluster weight to obtain the medical image cluster includes:

[0022] Calculating an image distribution probability that the target medical image belongs to the target Gaussian distribution according to the image pixel mean, the target cluster weight and the target Gaussian distribution;

[0023] Screening the target Gaussian distribution according to the image distribution probability to obtain a reference Gaussian distribution;

[0024] The target medical image is divided into the reference Gaussian distribution to obtain the medical image cluster.

[0025] In some embodiments, the training of the source medical image classification model according to the medical image cluster to obtain a reference classification model includes:

[0026] Initialize the model according to the source medical image classification model to obtain an original classification model;

[0027] Performing image classification on the target medical image by using the original classification model to obtain a pseudo label;

[0028] Perform a first loss calculation based on the pseudo-label to obtain model loss data;

[0029] Perform a second loss calculation based on the pseudo-label to obtain information entropy loss data;

[0030] Summing the model loss data and the information entropy loss data to obtain second target loss data;

[0031] The model parameters of the original classification model are adjusted according to the second target loss data to obtain the reference classification model.

[0032] In some embodiments, clustering the target medical image according to the image pixel means to obtain medical image clusters and target cluster weights of the medical image clusters includes:

[0033] Calculating the pixel distance between the image pixel mean and a preset cluster center;

[0034] Screening the preset cluster centers according to the pixel distance to obtain a target cluster center;

[0035] Dividing the target medical image into the target cluster center to obtain the medical image cluster;

[0036] The target cluster weight is determined according to the image pixel mean of the target medical image in the medical image cluster.

[0037] In some embodiments, performing model training according to the source medical image and the disease category label to obtain a source medical image classification model includes:

[0038] Predicting the disease category of the source medical image by using a preset classification model to obtain a predicted disease category;

[0039] Calculate the loss according to the predicted disease category and the disease category label to obtain source loss data;

[0040] The model parameters of the preset classification model are updated according to the source loss data to obtain the source medical image classification model.

[0041] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a medical image classification device, the device comprising:

[0042] A first medical data terminal is used to store a source medical image and a disease category label of the source medical image, and perform model training according to the source medical image and the disease category label to obtain a source medical image classification model;

[0043] The second medical data terminal is used to store the target medical image, perform model training according to the target medical image and the source medical image classification model to obtain the target medical image classification model, and classify the target medical image through the target medical image classification model to obtain the target disease category.

[0044] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0045] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0046] The medical image classification method, medical image classification device, electronic device and computer-readable storage medium proposed in the embodiments of the present application are different in data distribution and annotation of medical images between different medical institutions. By training the model at the first medical data end according to its local source medical image and disease category label, it not only helps to protect data privacy and security, but also reduces the model deviation caused by data distribution differences and annotation differences, and obtains the source medical image classification model, thereby improving the accuracy and reliability of the source medical image classification model. At the second medical data end, the model is trained according to the target medical image and the source medical image classification model, and the medical image classification knowledge learned by the source medical image classification model can be applied to the target medical image, thereby improving the generalization ability and classification accuracy of the image classification model, and there is no need to annotate the target medical image to obtain the target medical image classification model. The target medical image is classified by the target medical image classification model to obtain the target disease category, which solves the problems of data distribution mismatch and annotation difficulties and reduces the difficulty of medical image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of a medical image classification method provided in an embodiment of the present application;

[0048] Figure 2 yes Figure 1Flow chart of step S110 in FIG.

[0049] Figure 3 yes Figure 1 Flow chart of step S120 in FIG.

[0050] Figure 4 yes Figure 3 Flow chart of step S320 in FIG.

[0051] Figure 5 yes Figure 4 Flow chart of step S480 in FIG.

[0052] Figure 6 yes Figure 3 Flow chart of step S320 in FIG.

[0053] Figure 7 yes Figure 3 Flow chart of step S330 in FIG.

[0054] Figure 8 is a schematic diagram of the structure of a medical image classification device provided in an embodiment of the present application;

[0055] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0059] The training of medical image classification models requires a large number of medical image samples, but the data distribution and annotation of medical image samples vary between different medical institutions. For example, the shapes and brightness of tissues and organs photographed by different medical institutions are different, and the imaging modes of medical image samples are different. In addition, labeling medical image samples consumes a lot of time, which makes medical image classification more difficult.

[0060] Based on this, the embodiments of the present application provide a medical image classification method, a medical image classification device, an electronic device and a computer-readable storage medium, aiming to reduce the difficulty of medical image classification.

[0061] The medical image classification method, medical image classification device, electronic device and computer-readable storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the medical image classification method in the embodiments of the present application is described.

[0062] The medical image classification method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The medical image classification method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the medical image classification method, etc., but is not limited to the above forms.

[0063] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0064] Figure 1It is an optional flow chart of the medical image classification method provided in an embodiment of the present application. The medical image classification method is applied to a medical image classification device. The medical image classification device includes a first medical data terminal and a second medical data terminal. The first medical data terminal stores an active medical image and a disease category label of the source medical image, and the second medical data terminal stores a target medical image. The medical image classification method may include but is not limited to steps S110 to S120.

[0065] Step S110, at the first medical data end, model training is performed according to the source medical image and the disease category label to obtain a source medical image classification model;

[0066] Step S120, at the second medical data end, model training is performed based on the target medical image and the source medical image classification model to obtain a target medical image classification model, and image classification is performed on the target medical image through the target medical image classification model to obtain a target disease category.

[0067] Steps S110 to S120 shown in the embodiment of the present application can solve the problems of data distribution mismatch and labeling difficulty by transferring the medical image classification knowledge learned by the source medical image classification model to the classification process of the target medical image, thereby reducing the difficulty of image classification.

[0068] The medical data terminal refers to a system or platform used for data collection, storage, processing, analysis and application in the medical field. The first medical data terminal and the second medical data terminal are different data management and application platforms. Due to differences in factors such as image acquisition conditions, equipment scanning parameters, patient groups, image annotation standards, etc., there are significant differences in the data distribution and annotation of medical images between the first medical data terminal and the second medical data terminal. The source medical image and the target medical image are images generated in the medical field using various imaging technologies such as computed tomography and magnetic resonance imaging. The source medical image is stored in the first medical data terminal, and the target medical image is stored in the second medical data terminal. The target medical image is an unlabeled medical image. In the model training process, image data distribution and annotation differences are important factors affecting the model classification performance. In order to improve the classification performance of the model, the first medical data terminal uses the locally stored source medical images and the disease category labels of the source medical images to train the model and obtain the source medical image classification model. The disease category label refers to a label used to identify and distinguish different disease categories. The disease category label includes a single label or multiple labels. For example, a chest X-ray can be labeled with a single label of pneumonia, or it can be labeled with two labels of pneumonia and tuberculosis.

[0069] See also Figure 2 In some embodiments, step S110 may include but is not limited to steps S210 to S230:

[0070] Step S210, predicting the disease category of the source medical image by using a preset classification model to obtain a predicted disease category;

[0071] Step S220, performing loss calculation according to the predicted disease category and the disease category label to obtain source loss data;

[0072] Step S230, updating the model parameters of the preset classification model according to the source loss data to obtain a source medical image classification model.

[0073] In step S210 of some embodiments, the predicted disease category is the disease category output by the preset classification model for the source medical image. The preset classification model adopts an encoding-decoding structure, the preset classification model includes an encoder and a decoder, the encoder includes a convolution sub-model and an attention sub-model, and the source medical image is subjected to multi-scale feature extraction by the convolution sub-model to extract features at different levels to obtain a first image feature and a second image feature, and the feature size of the first image feature is greater than the feature size of the second image feature. The first image feature and the second image feature are both local feature representations, which may be edge features, texture features, structure and shape features of tissues and organs, etc. In order to extract the global context feature representation, the second image feature is subjected to feature extraction by the attention sub-model to capture the dependency between the regions of the medical image to obtain the attention feature. The attention sub-model adopts a transformer architecture. The second image feature and the attention feature are subjected to feature fusion to combine the local feature representation and the global feature representation to obtain a richer feature representation and obtain the original fused feature. In order to achieve accurate classification of the medical image, a jump connection is set between the convolution sub-model and the decoder to retain the detailed features of the image. The original fused feature is decoded by the decoder to obtain a first decoded feature, the first decoded feature is fused with the second image feature by a jump connection to obtain a first fused feature, and the first fused feature and the first image feature are fused by a jump connection to combine diverse features of different scales to obtain a second fused feature. The disease category is predicted based on the second fused feature to obtain a predicted disease category.

[0074] In step S220 of some embodiments, in order to determine the difference between the predicted disease category and the disease category label, the predicted disease category and the disease category label are subjected to loss calculation based on the loss function to obtain source loss data. s samples, the source dataset is represented as is the i-th source medical image, is the disease category label of the i-th source medical image. The preset classification model is represented by f s , K disease categories can be classified. The loss function is defined as:

[0075]

[0076] Among them, L s represents source loss data; E represents the expected function; x s is the source medical image; s is the disease category label of the source medical image; f s is a preset classification model, the output of which is the predicted disease category, indicating the probability that the source medical image belongs to the kth disease category; δ k is the softmax function.

[0077] The softmax function is expressed as:

[0078]

[0079] Among them, a is the input of the softmax function.

[0080] In step S230 of some embodiments, the source loss data is minimized and the model parameters of the preset classification model are updated to optimize the preset classification model to obtain a source medical image classification model.

[0081] In the above steps S210 to S230, the model is trained through local samples of the second medical data terminal, so that the model can fully learn the medical image distribution of the second medical data terminal, thereby improving the classification accuracy of the model for local medical images.

[0082] Considering the privacy and security of medical image data, the second medical data terminal cannot access the source medical image and the feature space of the source medical image of the first medical data terminal, and medical image labeling is difficult. Therefore, the embodiment of the present application performs model training at the second medical data terminal based on the target medical image and the source medical image classification model, and transfers the medical image classification knowledge learned from the source medical image by the source medical image classification model to the image classification of the second medical data terminal without using the source medical image and labeling the target medical image, thereby obtaining the target medical image classification model. The embodiment of the present application completes the training of the target medical image classification model based on a small number of labeled source medical images and a large number of unlabeled target medical images, thereby reducing the labeling cost of medical images. The target medical image is classified by the target medical image classification model, the disease category of the target medical image is determined, and the target disease category is obtained.

[0083] See also Figure 3 In some embodiments, step S120 may include but is not limited to steps S310 to S340:

[0084] Step S310, performing mean calculation on the target medical image to obtain the image pixel mean;

[0085] Step S320, clustering the target medical image according to the image pixel mean value to obtain medical image clusters and target cluster weights of the medical image clusters;

[0086] Step S330, training the source medical image classification model according to the medical image cluster to obtain a reference classification model;

[0087] Step S340, performing model aggregation on the reference classification model according to the target cluster weight to obtain a target medical image classification model.

[0088] In step S310 of some embodiments, the pixel value of each pixel in the target medical image is obtained, and the pixel values ​​of all pixels are averaged to obtain the image pixel mean. The image pixel mean can reflect the overall brightness and grayscale distribution of the image, which helps to identify the grayscale characteristics of different tissues in the medical image. The larger the image pixel mean, the brighter the image as a whole, and the more pixels with higher grayscale values ​​in the image. The method of mean calculation is expressed as:

[0089]

[0090] Where I represents the target medical image, p x is the number of pixels in the target medical image; (x i ,y i ) is the image space coordinate of the i-th pixel in the target medical image, x i is the horizontal coordinate of the i-th pixel, y i is the ordinate of the i-th pixel; I(x i ,y i ) is the pixel value of the i-th pixel.

[0091] In step S320 of some embodiments, the second medical data terminal includes medical images of different modalities, different types of lesions and tissues. In order to distinguish different medical images, the target medical images are clustered by the image pixel mean, and the target medical images with greater similarity are classified into the same cluster, and the target medical images with less similarity are classified into different clusters to obtain medical image clusters, and the target cluster weights of the medical image clusters are calculated. The target cluster weights are used to indicate the contribution of the medical image clusters to the model training.

[0092] In step S330 of some embodiments, different medical image clusters can provide different types of lesions and tissue information. In order to improve the model's ability to classify different medical images, the source medical image classification model is used as an initialized model and trained according to the medical image clusters, so that the source medical image classification model can focus on different image features of different medical image clusters to learn the data features of medical images from different angles and obtain a reference classification model. In addition, different reference classification models can be trained in parallel using different medical image clusters, thereby improving the efficiency of model training.

[0093] In step S340 of some embodiments, different reference classification models may focus on different areas or features of the medical image. For example, a certain reference classification model focuses on detecting tumor areas, while other reference classification models focus on detecting blood vessels or bone structures. In order to improve the comprehensive feature learning ability of the model, the target cluster weight is multiplied by the reference classification model, and the multiplied models are added together to obtain the target medical image classification model.

[0094] The second medical data terminal includes medical image cluster 1 and medical image cluster 2, the target cluster weight of medical image cluster 1 is w1, and the target cluster weight of medical image cluster 2 is w2. The reference classification model 1 is obtained by training the medical image cluster 1 and the source medical image classification model, and the reference classification model 2 is obtained by training the medical image cluster 2 and the source medical image classification model. w1 is multiplied by the model parameters of the reference classification model 1, w2 is multiplied by the model parameters of the reference classification model 2, and the multiplication results are added to obtain the target medical image classification model.

[0095] Through the above steps S310 to S340, the image features of the target medical image can be learned from different angles, thereby improving the model's learning ability for image features, and further improving the model's classification ability for medical images.

[0096] See also Figure 4 In some embodiments, step S320 may include but is not limited to steps S410 to S480:

[0097] Step S410, calculating the distribution probability value that the image pixel mean belongs to a preset Gaussian distribution according to the image pixel mean, the preset cluster weight, the preset Gaussian mean and the preset Gaussian variance; the preset Gaussian distribution is determined by the preset Gaussian mean and the preset Gaussian variance;

[0098] Step S420, updating a preset Gaussian mean according to the distribution probability value and the image pixel mean to obtain a reference Gaussian mean;

[0099] Step S430, updating the preset Gaussian variance according to the image pixel mean, the distribution probability value and the reference Gaussian mean to obtain the reference Gaussian variance;

[0100] Step S440, updating the preset cluster weight according to the distribution probability value to obtain a reference cluster weight;

[0101] Step S450, determining first target loss data according to the image pixel mean, the reference Gaussian mean, the reference Gaussian variance and the reference cluster weight;

[0102] Step S460, adjusting the reference cluster weight according to the first target loss data to obtain the target cluster weight;

[0103] Step S470, adjusting the reference Gaussian mean and the reference Gaussian variance according to the first target loss data to obtain a target Gaussian distribution;

[0104] Step S480 , dividing the target medical image according to the target Gaussian distribution and the target cluster weight to obtain medical image clusters.

[0105] In step S410 of some embodiments, in order to improve the generalization ability of the image classification model and avoid model overfitting, the embodiments of the present application set the medical image samples used for model training to obey the Gaussian distribution. The preset cluster weight, the preset Gaussian mean, and the preset Gaussian variance are all randomly initialized values. The preset Gaussian mean and the preset Gaussian variance determine the preset Gaussian distribution, the preset Gaussian mean is the center of the preset Gaussian distribution, and the preset Gaussian variance is used to describe the shape of the preset Gaussian distribution. In order to obtain the Gaussian distribution obeyed by the target medical image, the probability that the image pixel mean belongs to the preset Gaussian distribution is calculated to obtain the distribution probability value. The calculation formula is defined as:

[0106]

[0107] Where M is the number of preset Gaussian distributions; m i is the image pixel mean of the i-th target medical image; p ik is the distribution probability value that the image pixel mean of the i-th target medical image belongs to the k-th preset Gaussian distribution; w k is the preset cluster weight of the kth preset Gaussian distribution; N(μ k ,σ k ) is the preset Gaussian distribution, μ k is the preset Gaussian mean of the kth preset Gaussian distribution, σ k is the preset Gaussian variance of the kth preset Gaussian distribution; N(m i |μ k ,σ k ) is to convert m i Substitute the output value calculated by the probability density function of the preset Gaussian distribution.

[0108] In step S420 of some embodiments, the parameters of the preset Gaussian distribution are optimized to accurately calculate the probability that the image pixel mean belongs to a specific Gaussian distribution, so that the target medical image can fit the specific Gaussian distribution well, which helps the model adapt to the data characteristics of different target medical images, thereby improving the model's ability to distinguish different medical image regions.

[0109] The preset Gaussian mean of the preset Gaussian distribution is updated according to the distribution probability value and the image pixel mean to obtain a reference Gaussian mean. The updating method of the preset Gaussian mean is expressed as:

[0110]

[0111] Among them, μ′ k is the reference Gaussian mean after the kth preset Gaussian mean is updated; N t is the number of target medical images; m i is the image pixel mean of the i-th target medical image; p ik is the distribution probability value that the image pixel mean of the i-th target medical image belongs to the k-th preset Gaussian distribution.

[0112] In step S430 of some embodiments, a preset Gaussian variance of a preset Gaussian distribution is updated according to the image pixel mean, the distribution probability value and the reference Gaussian mean to obtain a reference Gaussian variance. The updating method of the preset Gaussian variance is expressed as:

[0113]

[0114] Among them, σ′ k is the reference Gaussian variance obtained by updating the kth preset Gaussian variance; m i is the image pixel mean of the i-th target medical image; p ik is the distribution probability value that the image pixel mean of the i-th target medical image belongs to the k-th preset Gaussian distribution; N t is the number of target medical images; μ′ k is the reference Gaussian mean; T represents the transpose operation.

[0115] In step S440 of some embodiments, the mean of the distribution probability values ​​of the mean values ​​of each image pixel belonging to the preset Gaussian distribution is calculated to update the preset cluster weights to obtain the reference cluster weights. The updating method of the preset cluster weights is expressed as:

[0116]

[0117] Among them, w′ k is the reference cluster weight obtained by updating the preset cluster weight corresponding to the kth preset Gaussian distribution; p ikis the distribution probability value that the image pixel mean of the i-th target medical image belongs to the k-th preset Gaussian distribution; N t is the number of target medical images.

[0118] In step S450 of some embodiments, a reference Gaussian distribution is determined based on a reference Gaussian mean and a reference Gaussian variance. In order to determine whether the image pixel mean obeys the reference Gaussian distribution, a loss calculation is performed based on the image pixel mean, the reference Gaussian mean, the reference Gaussian variance and the reference cluster weight to determine the first target loss data. The formula for loss calculation is defined as:

[0119]

[0120] Among them, w′ j is the reference cluster weight corresponding to the jth reference Gaussian distribution; μ′ j is the reference Gaussian mean of the jth reference Gaussian distribution; σ′ j is the reference Gaussian variance of the jth reference Gaussian distribution; m i is the image pixel mean of the i-th target medical image; N t is the number of target medical images; M is the number of reference Gaussian distributions; N(m i |μ′ j ,σ′ j ) is to convert m i Substitute the output value calculated by the probability density function of the reference Gaussian distribution; ∏ represents the multiplication operator; ∑ represents the addition operator.

[0121] In steps S460 to S470 of some embodiments, in order to obtain the Gaussian distribution obeyed by the image pixel mean, the first target loss data is minimized to adjust the reference cluster weight to obtain the target cluster weight, and the first target loss data is minimized to adjust the reference Gaussian mean and the reference Gaussian variance to obtain the target Gaussian distribution. The target Gaussian distribution includes a target Gaussian mean and a target Gaussian variance, the target Gaussian mean is obtained by updating the reference Gaussian mean, and the target Gaussian variance is obtained by updating the reference Gaussian variance.

[0122] In step S480 of some embodiments, the target medical image is divided according to the target Gaussian distribution and the target cluster weight, the target medical images that obey the target Gaussian distribution are divided into the same cluster, and the target medical images that do not obey the target Gaussian distribution are divided into different clusters, to obtain medical image clusters.

[0123] Through the above steps S410 to S480, the target medical image in the medical image cluster can be made to obey the Gaussian distribution, thereby improving the generalization of the reference classification model, so that the reference classification model can accurately classify diseases for unknown target medical images.

[0124] See also Figure 5 In some embodiments, step S480 may include but is not limited to steps S510 to S530:

[0125] Step S510, calculating the image distribution probability that the target medical image belongs to the target Gaussian distribution according to the image pixel mean, the target cluster weight and the target Gaussian distribution;

[0126] Step S520, screening the target Gaussian distribution according to the image distribution probability to obtain a reference Gaussian distribution;

[0127] Step S530 , dividing the target medical image into reference Gaussian distributions to obtain medical image clusters.

[0128] In step S510 of some embodiments, referring to step S410, the probability that the image pixel mean of the target medical image belongs to the target Gaussian distribution is calculated based on the image pixel mean, the target cluster weight and the target Gaussian distribution to obtain the image distribution probability.

[0129] In step S520 of some embodiments, in order to obtain a Gaussian distribution that can accurately fit the target medical image, the target Gaussian distribution with the highest image distribution probability is selected as the reference Gaussian distribution.

[0130] In step S530 of some embodiments, a corresponding set is set for each Gaussian distribution, and the target medical image is divided into a set of reference Gaussian distributions to obtain a medical image cluster.

[0131] Through the above steps S510 to S530, a Gaussian distribution that accurately fits the target medical image can be obtained to accurately divide the target medical image.

[0132] See also Figure 6 In some embodiments, step S320 may include but is not limited to steps S610 to S640:

[0133] Step S610, calculating the pixel distance between the image pixel mean and the preset cluster center;

[0134] Step S620, screening the preset cluster centers according to the pixel distance to obtain the target cluster center;

[0135] Step S630, dividing the target medical image into target cluster centers to obtain medical image clusters;

[0136] Step S640 , determining a target cluster weight according to the image pixel mean of the target medical image in the medical image cluster.

[0137] In step S610 of some embodiments, a preset cluster center is initialized, and the Euclidean distance between the image pixel mean and the preset cluster center is calculated to obtain the pixel distance. The larger the pixel distance, the smaller the similarity between the image pixel mean and the preset cluster center.

[0138] In step S620 of some embodiments, the preset cluster center with the smallest pixel distance is selected as the original cluster center, and the target medical image is divided into the original cluster center to obtain the cluster center with greater similarity to the image pixel mean, and obtain the original cluster. The image pixel mean of the target medical image in the original cluster is averaged to obtain the target pixel mean. The formula for average calculation is expressed as:

[0139]

[0140] Where R is the number of target medical images in the original cluster; m i is the image pixel mean of the i-th target medical image in the original cluster.

[0141] Update the original cluster center according to the target pixel mean to obtain the reference cluster center. If the reference cluster center does not change much for multiple consecutive times or the number of updates reaches the preset threshold, the reference cluster center is used as the benchmark cluster center. Calculate the pixel distance between the image pixel mean and the benchmark cluster center, and select the benchmark cluster center with the smallest pixel distance as the target cluster center.

[0142] In step S630 of some embodiments, the target medical image corresponding to the image pixel mean is divided into target cluster centers to obtain a medical image cluster.

[0143] In step S640 of some embodiments, weight calculation is performed based on the image pixel mean of the target medical image in the medical image cluster to determine the target cluster weight. The formula for weight calculation is expressed as:

[0144]

[0145] Among them, q i is the target cluster weight of the i-th medical image cluster; Q is the number of medical image clusters; N i is the number of target medical images in the i-th medical image cluster; m ij is the image pixel mean of the jth target medical image in the i-th medical image cluster.

[0146] Through the above steps S610 to S640, the target medical images with similar grayscale distribution can be divided into the same cluster, so that the model can learn common features from medical image samples with consistent grayscale distribution, capture key features related to specific disease classification, and improve the classification ability of the reference classification model for specific disease categories.

[0147] See also Figure 7 In some embodiments, step S330 may include but is not limited to steps S710 to S760:

[0148] Step S710, initializing the model according to the source medical image classification model to obtain an original classification model;

[0149] Step S720, classifying the target medical image using the original classification model to obtain a pseudo label;

[0150] Step S730, performing a first loss calculation based on the pseudo-label to obtain model loss data;

[0151] Step S740, performing a second loss calculation based on the pseudo-label to obtain information entropy loss data;

[0152] Step S750, summing the model loss data and the information entropy loss data to obtain second target loss data;

[0153] Step S760, adjusting the model parameters of the original classification model according to the second target loss data to obtain a reference classification model.

[0154] In step S710 of some embodiments, the source medical image classification model of the first medical data end is used as the initialization model of the second medical data end to obtain the original classification model.

[0155] In step S720 of some embodiments, at the second medical data end, the unlabeled target data set includes N t target medical images, the target dataset is represented as Represents the i-th target medical image. The target medical image has no label. The target medical image is input into the original classification model for image classification to obtain the pseudo label of the target medical image. The pseudo label is used to indicate the disease category predicted by the original classification model for the target medical image.

[0156] In step S730 of some embodiments, in order to obtain the classification loss of the original classification model on the target medical image, a first loss calculation is performed based on the pseudo label to obtain model loss data. The formula for the first loss calculation is expressed as:

[0157]

[0158] Among them, L t represents the model loss data; E represents the expectation; x t represents the target medical image; D T represents the target dataset; K represents the number of disease categories; f t represents the original classification model, f t(x t ) is the pseudo label output by the original classification model; δ k is the softmax function.

[0159] In step S740 of some embodiments, in order to match the feature space of the target data set with the feature space of the source data set, a second loss calculation is performed based on the pseudo-labels based on the information entropy maximization theory to obtain information entropy loss data. The information entropy loss data is used to evenly distribute and disperse the feature space of the target data set on each disease category.

[0160] The formula for calculating the second loss is expressed as:

[0161]

[0162] Among them, L info represents information entropy loss data; K represents the number of predictable disease categories; represents the average output vector of the target dataset in the kth disease category; E represents the expectation; D T represents the target dataset; f t represents the original classification model, f t (x t ) is the pseudo label output by the original classification model; δ k is the softmax function.

[0163] In step S750 of some embodiments, in order to measure the overall model performance of the original classification model, the model loss data and the information entropy loss data are added to obtain the second target loss data.

[0164] In step S760 of some embodiments, the model parameters of the original classification model are adjusted to optimize the original classification model. When the second target loss data is the smallest, the absolute value of the difference between multiple consecutive second target loss data is less than a preset error threshold, or the number of iterations reaches a preset iteration threshold, the original classification model at this time is used as a reference classification model.

[0165] The above steps S710 to S760 perform model training through the source medical image classification model and the target medical image, so that the source medical image classification model can learn a wider range of image feature representations on the target medical image, so that the reference classification model can adapt to the data distribution of the second medical data end, and solve the problem of difficulty in medical image classification caused by large differences in medical image distribution.

[0166] See also Figure 8 The present application also provides a medical image classification device, which can implement the above-mentioned medical image classification method. The medical image classification device includes:

[0167] The first medical data terminal 810 is used to store the source medical image and the disease category label of the source medical image, and perform model training according to the source medical image and the disease category label to obtain a source medical image classification model;

[0168] The second medical data terminal 820 is used to store the target medical image, perform model training according to the target medical image and the source medical image classification model to obtain a target medical image classification model, and perform image classification on the target medical image through the target medical image classification model to obtain a target disease category.

[0169] The specific implementation of the medical image classification device is basically the same as the specific implementation of the above-mentioned medical image classification method, and will not be repeated here.

[0170] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned medical image classification method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0171] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0172] The processor 910 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0173] The memory 920 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 920 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 920, and the processor 910 calls and executes the medical image classification method of the embodiment of this application;

[0174] Input / output interface 930, used to implement information input and output;

[0175] Communication interface 940, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0176] bus 950 , which transmits information between the various components of the device (e.g., processor 910 , memory 920 , input / output interface 930 , and communication interface 940 );

[0177] The processor 910 , the memory 920 , the input / output interface 930 , and the communication interface 940 are connected to each other in communication within the device via a bus 950 .

[0178] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned medical image classification method is implemented.

[0179] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0180] The medical image classification method, medical image classification device, electronic device and computer storage medium provided in the embodiments of the present application can solve the problems of data distribution mismatch and labeling difficulty by transferring the medical image classification knowledge learned by the source medical image classification model to the classification process of the target medical image, thereby reducing the difficulty of image classification.

[0181] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0182] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0183] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0184] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0185] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0186] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0187] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0188] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0189] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0190] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0191] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A medical image classification method, characterized in that: Applied to a medical image classification device, the medical image classification device includes a first medical data terminal and a second medical data terminal, the first medical data terminal stores a source medical image and a disease category label of the source medical image, and the second medical data terminal stores a target medical image, the method includes: At the first medical data end, model training is performed according to the source medical image and the disease category label to obtain a source medical image classification model; At the second medical data end, model training is performed based on the target medical image and the source medical image classification model to obtain a target medical image classification model, and the target medical image is classified by the target medical image classification model to obtain a target disease category.

2. The method according to claim 1, characterized in that The performing model training according to the target medical image and the source medical image classification model to obtain the target medical image classification model includes: Performing mean calculation on the target medical image to obtain an image pixel mean; Clustering the target medical image according to the image pixel mean to obtain a medical image cluster and a target cluster weight of the medical image cluster; Training the source medical image classification model according to the medical image cluster to obtain a reference classification model; The reference classification model is model aggregated according to the target cluster weight to obtain the target medical image classification model.

3. The method according to claim 2, characterized in that The step of clustering the target medical image according to the image pixel mean to obtain a medical image cluster and a target cluster weight of the medical image cluster includes: Calculating a distribution probability value that the image pixel mean belongs to a preset Gaussian distribution according to the image pixel mean, the preset cluster weight, the preset Gaussian mean and the preset Gaussian variance; the preset Gaussian distribution is determined by the preset Gaussian mean and the preset Gaussian variance; Update the preset Gaussian mean according to the distribution probability value and the image pixel mean to obtain a reference Gaussian mean; Update the preset Gaussian variance according to the image pixel mean, the distribution probability value and the reference Gaussian mean to obtain a reference Gaussian variance; Update the preset cluster weight according to the distribution probability value to obtain a reference cluster weight; Determining first target loss data according to the image pixel mean, the reference Gaussian mean, the reference Gaussian variance and the reference cluster weight; Adjust the reference cluster weight according to the first target loss data to obtain the target cluster weight; Adjust the reference Gaussian mean and the reference Gaussian variance according to the first target loss data to obtain a target Gaussian distribution; The target medical image is divided into image clusters according to the target Gaussian distribution and the target cluster weights to obtain the medical image clusters.

4. The method according to claim 3, characterized in that The step of performing image segmentation on the target medical image according to the target Gaussian distribution and the target cluster weight to obtain the medical image cluster includes: Calculating an image distribution probability that the target medical image belongs to the target Gaussian distribution according to the image pixel mean, the target cluster weight and the target Gaussian distribution; Screening the target Gaussian distribution according to the image distribution probability to obtain a reference Gaussian distribution; The target medical image is divided into the reference Gaussian distribution to obtain the medical image cluster.

5. The method according to claim 2, characterized in that: The step of training the source medical image classification model according to the medical image cluster to obtain a reference classification model includes: Initialize the model according to the source medical image classification model to obtain an original classification model; Performing image classification on the target medical image by using the original classification model to obtain a pseudo label; Perform a first loss calculation based on the pseudo-label to obtain model loss data; Perform a second loss calculation based on the pseudo-label to obtain information entropy loss data; Summing the model loss data and the information entropy loss data to obtain second target loss data; The model parameters of the original classification model are adjusted according to the second target loss data to obtain the reference classification model.

6. The method according to claim 2, characterized in that The step of clustering the target medical image according to the image pixel mean to obtain a medical image cluster and a target cluster weight of the medical image cluster includes: Calculating the pixel distance between the image pixel mean and a preset cluster center; Screening the preset cluster centers according to the pixel distance to obtain a target cluster center; Dividing the target medical image into the target cluster center to obtain the medical image cluster; The target cluster weight is determined according to the image pixel mean of the target medical image in the medical image cluster.

7. The method according to any one of claims 1 to 6, characterized in that: The performing model training according to the source medical image and the disease category label to obtain a source medical image classification model includes: Predicting the disease category of the source medical image by using a preset classification model to obtain a predicted disease category; Calculate the loss according to the predicted disease category and the disease category label to obtain source loss data; The model parameters of the preset classification model are updated according to the source loss data to obtain the source medical image classification model.

8. A medical image classification device, characterized in that: The device comprises: A first medical data terminal is used to store a source medical image and a disease category label of the source medical image, and perform model training according to the source medical image and the disease category label to obtain a source medical image classification model; The second medical data terminal is used to store the target medical image, perform model training according to the target medical image and the source medical image classification model to obtain the target medical image classification model, and classify the target medical image through the target medical image classification model to obtain the target disease category.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images