Auxiliary diagnosis model training method and device

By building a training data set containing client medical images and public images, the feature information of the auxiliary diagnostic model is aligned and joint training is carried out on the server, the problems of insufficient samples and data privacy protection in medical institutions are solved, and efficient auxiliary is achieved Diagnostic model training and data sharing.

CN119028561APending Publication Date: 2024-11-26TSINGHUA UNIVERSITY +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310591768.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-23
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The insufficient number of medical samples in medical institutions makes it difficult to train deep learning models, and due to privacy protection reasons, it is difficult to share data between different institutions, limiting the effects of federated learning and transfer learning.

Method used

By building a training data set containing client medical images and public images, the auxiliary diagnostic model is iteratively trained using optimization functions to align the feature information distribution of the model and send the model and identification information to the server to build the training data on the server set, realize joint training of the model.

Benefits of technology

Reduces the difficulty of training auxiliary diagnostic model, allows the use of public image assisted training with greater differences from medical images, increases the amount of training data, improves the training effect of the model, and realizes the model while ensuring data privacy protection. Collaborative training.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119028561A_ABST
    Figure CN119028561A_ABST
Patent Text Reader

Abstract

The invention discloses an auxiliary diagnosis model training method and device. The method comprises the following steps: constructing a first training data set for training a first auxiliary diagnosis model; inputting samples in the first training data set into the first auxiliary diagnosis model, and performing iterative training on the first auxiliary diagnosis model through a first optimization function to enable the first auxiliary diagnosis model to meet a first optimization target; and sending the first auxiliary diagnosis model and identification information for identifying the public image to a server, so that the server trains a second auxiliary diagnosis model based on the identification information and the first auxiliary diagnosis model, and the second auxiliary diagnosis model conforms to a second optimization target. According to the method, a better training effect can be formed, and the training difficulty of a large-size auxiliary diagnosis model can be reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to an auxiliary diagnosis model training and an auxiliary diagnosis model training device. Background Art

[0002] Computer-Aided Diagnosis (CAD) based on deep learning (DL) has achieved excellent results in medical image analysis (such as X-ray image classification). As a form of auxiliary diagnosis, it can help radiologists improve diagnostic accuracy and consistency in image and disease interpretation, and alleviate delayed diagnosis and misdiagnosis caused by large case differences and physician fatigue.

[0003] The training of deep learning models depends on a sufficiently large labeled data set. However, the number of medical samples in a single medical institution is small, which is usually difficult to meet the training needs of deep learning models. Even if large medical institutions have a large number of medical samples, medical experts usually find it difficult to label the medical samples in sufficient quantity and in a timely manner. Therefore, the medical samples of a single medical institution are usually difficult to meet the training needs of deep learning models. If the labeled medical samples of different medical institutions can be shared, the number of labeled samples can be effectively increased. However, due to the privacy protection requirements of patients, institutions and countries, some medical data are usually not open to the public and cannot be shared between different medical institutions.

[0004] Federated Learning (FL) is a distributed machine learning technology that emphasizes data security and privacy protection. It enables decentralized clients to collaboratively train machine learning models without disclosing private data by sharing machine learning model parameters. However, conventional federated learning methods are still limited by the small number of medical samples in medical institutions, which results in the inability to fit the auxiliary diagnosis model of the medical institution's client, affecting the training effect of the joint training model on the server.

[0005] Transfer learning can use existing knowledge to learn new knowledge by finding similarities between existing knowledge and new knowledge. In the specific auxiliary diagnosis model training scenario, transfer learning can use public samples to assist the client's auxiliary diagnosis model to learn medical samples. Conventional transfer learning methods usually transfer in similar fields. If the similarity between existing knowledge and new knowledge is not enough, the migration result is not ideal. Therefore, it is usually necessary to use public and similar medical data to assist the client's auxiliary diagnosis model training. However, due to the privacy attributes of medical data, it is usually difficult to obtain a sufficient amount of medical data in public datasets, and the authenticity and data quality of public medical data are usually difficult to guarantee. Summary of the invention

[0006] In view of the above-mentioned problems existing in the prior art, the present application provides an auxiliary diagnosis model training method and an auxiliary diagnosis model training device.

[0007] The first aspect of the present application provides an auxiliary diagnosis model training method, which is applied to a client, comprising:

[0008] Constructing a first training data set for training a first auxiliary diagnosis model; the first auxiliary diagnosis model includes a first feature extraction network and a first classification network; the first training data set includes a medical image of a client, a label corresponding to the medical image, a public image selected from a public gallery that matches the medical image, and a label corresponding to the public image, wherein the label is used to identify an auxiliary diagnosis result;

[0009] Input samples in the first training data set into the first auxiliary diagnosis model, and iteratively train the first auxiliary diagnosis model through a first optimization function, so that the first auxiliary diagnosis model meets a first optimization goal; the first optimization goal at least includes aligning feature distributions of first feature information and second feature information, the first feature information includes feature information extracted from the medical image by the first feature extraction network, and the second feature information includes feature information extracted from the public image by the first feature extraction network;

[0010] The first auxiliary diagnosis model and identification information for identifying the public image are sent to the server, so that the server trains a second auxiliary diagnosis model based on the identification information and the first auxiliary diagnosis model, so that the second auxiliary diagnosis model meets the second optimization goal; the identification information is used by the server to obtain the public image and construct a second training data set; the second auxiliary diagnosis model includes a second feature extraction network and a second classification network; the second optimization goal at least includes aligning the feature distribution of third feature information and fourth feature information, the third feature information includes the feature information extracted by the server from the public image based on the first feature extraction network, and the fourth feature information includes the feature information extracted by the server from the public image based on the second feature extraction network.

[0011] In some embodiments, constructing a first training data set for training a first auxiliary diagnosis model includes:

[0012] Acquire medical data of the client, where the medical data includes a first medical image and a label corresponding to the first medical image;

[0013] Selecting a public image matching the first medical image from a public image library, and establishing a corresponding relationship between the public image and the label of the first medical image;

[0014] The first training data set is constructed based on the first medical image, the public image and the label.

[0015] In some embodiments, constructing a first training data set for training a first auxiliary diagnosis model includes:

[0016] Acquire medical data of the client, the medical data including a first medical image, a second medical image, and a label corresponding to the first medical image;

[0017] Based on the first auxiliary diagnosis model, selecting a third medical image whose confidence meets a first threshold from the second medical image, and generating a label corresponding to the third medical image;

[0018] Selecting a public image matching the first medical image from a public image library, and establishing a corresponding relationship between the public image and the label of the first medical image;

[0019] The first training data set is constructed based on the first medical image, the third medical image, the public image and the label.

[0020] In some embodiments, the constructing of a first training data set for training a first auxiliary diagnosis model further includes:

[0021] Perform data enhancement processing on the first medical image and / or the third medical image to increase the number of samples in the first training data set.

[0022] In some embodiments, selecting a public image matching the first medical image from a public image library includes:

[0023] Comparing a class of first medical images with the same label with various types of public images in the public image library to determine the similarity of data distribution between the class of first medical images and various types of public images;

[0024] The public image whose similarity meets the first preset condition is selected as the public image that matches the first medical image of this type.

[0025] In some embodiments, comparing a class of first medical images having the same label with various types of public images in the public image library to determine the similarity of data distribution between the class of first medical images and various types of public images includes:

[0026] Based on a class of first medical images having the same label, generating a first data matrix;

[0027] Based on a type of public images in the public database, generating a second data matrix;

[0028] Concatenating and transposing the first data matrix and the second data matrix to form a joint matrix;

[0029] Performing non-negative matrix decomposition on the joint matrix to obtain a shared basis matrix and a coefficient matrix; wherein the coefficient matrix includes a first coefficient matrix and a second coefficient matrix, the first coefficient matrix and the second coefficient matrix are horizontally spliced ​​to form the coefficient matrix, the first coefficient matrix represents the contribution degree of the shared basis matrix to the first data matrix, and the second coefficient matrix represents the contribution degree of the shared basis matrix to the second data matrix;

[0030] Based on the similarity between the first coefficient matrix and the second coefficient matrix, the similarity between the first medical image of the type and the public image of the type is determined.

[0031] In some embodiments, the first training data set includes a source domain and a target domain, the source domain includes the public image and a label corresponding to the public image, and the target domain includes the first medical image, the third medical image, and labels corresponding to the first medical image and the third medical image;

[0032] The first optimization function is used to optimize the parameters of the first auxiliary diagnosis model, align the feature distribution of the feature information extracted by the first feature extraction network from the public image and the first medical image, align the feature distribution of the feature information extracted by the first feature extraction network from similar samples in the target domain, align the category centroids of similar samples in the source domain and the target domain, so as to align the feature distribution of the feature information extracted by the first feature extraction network from similar samples in the source domain and the target domain, so that the first auxiliary diagnosis model meets the first optimization goal; wherein, the similar samples are samples with the same labels.

[0033] In some embodiments, the method further comprises:

[0034] Obtaining the predicted probability of the public image by the server based on the second auxiliary diagnosis model;

[0035] Based on the predicted probability, updating the labels of the public images in the first training data set;

[0036] The first auxiliary diagnosis model is trained based on the updated first training data set.

[0037] The second aspect of the present application provides an auxiliary diagnosis model training method, which is applied to a server, comprising:

[0038] Obtain identification information and a first auxiliary diagnosis model trained by a client; the first auxiliary diagnosis model is trained based on a first training data set, the first training data set includes medical images of the client and public images selected from a public gallery and matching the medical images; the identification information is used to identify the public images in the first training data set; the first auxiliary diagnosis model includes a first feature extraction network and a first classification network, the first auxiliary diagnosis model meets a first optimization goal, the first optimization goal at least includes aligning feature distributions of first feature information and second feature information, the first feature information includes feature information extracted from the medical image by the first feature extraction network, and the second feature information includes feature information extracted from the public image by the first feature extraction network;

[0039] Based on the identification information, construct a second training data set, where the second training data set includes at least part of the public images in the first training data set;

[0040] The samples in the second training data set are input into a second auxiliary diagnosis model, and the second auxiliary diagnosis model is iteratively trained through a second optimization function, so that the second auxiliary diagnosis model meets the second optimization goal; the second auxiliary diagnosis model includes a second feature extraction network and a second classification network; the second optimization goal at least includes aligning the feature distribution of the third feature information and the feature distribution of the fourth feature information, the third feature information includes feature information extracted from the public images of the second training data set based on the first feature extraction network, and the fourth feature information includes feature information extracted from the public images of the second training data set based on the second feature extraction network.

[0041] The third aspect of the present application provides an auxiliary diagnosis model training device, comprising:

[0042] A first construction module is configured to construct a first training data set for training a first auxiliary diagnosis model; the first auxiliary diagnosis model includes a first feature extraction network and a first classification network; the first training data set includes a medical image of a client, a label corresponding to the medical image, a public image selected from a public gallery and matching the medical image, and a label corresponding to the public image, wherein the label is used to identify an auxiliary diagnosis result;

[0043] a first training module, configured to input samples in the first training data set into the first auxiliary diagnosis model, and iteratively train the first auxiliary diagnosis model through a first optimization function, so that the first auxiliary diagnosis model meets a first optimization goal; the first optimization goal at least includes aligning feature distributions of first feature information and second feature information, the first feature information includes feature information extracted from the medical image by the first feature extraction network, and the second feature information includes feature information extracted from the public image by the first feature extraction network;

[0044] The first communication module is configured to send the first auxiliary diagnosis model and identification information for identifying the public image to the server, so that the server trains a second auxiliary diagnosis model based on the identification information and the first auxiliary diagnosis model, so that the second auxiliary diagnosis model meets the second optimization goal; the identification information is used by the server to obtain the public image and construct a second training data set; the second auxiliary diagnosis model includes a second feature extraction network and a second classification network; the second optimization goal at least includes aligning the feature distribution of third feature information and fourth feature information, the third feature information includes feature information extracted by the server from the public image based on the first feature extraction network, and the fourth feature information includes feature information extracted by the server from the public image based on the second feature extraction network.

[0045] The auxiliary diagnosis model training method of the embodiment of the present application has low requirements on the technical field correlation between the public image and the medical image, allows the use of public images that are significantly different from the medical images to assist in the training of the first auxiliary diagnosis model and the second auxiliary diagnosis model, and can form a better training effect, which is beneficial to reducing the difficulty of training large-size auxiliary diagnosis models. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of the auxiliary diagnosis model training method of the first embodiment of the present application;

[0047] Figure 2 This is a schematic diagram of the training scenario of the first auxiliary diagnosis model;

[0048] Figure 3 A schematic diagram of a training scenario for the second auxiliary diagnosis model;

[0049] Figure 4 This is a flowchart of step S110 in the auxiliary diagnosis model training method of the first embodiment of the present application;

[0050] Figure 5 This is a flow chart of the auxiliary diagnosis model training method of the second embodiment of the present application;

[0051] Figure 6 This is a structural block diagram of an auxiliary diagnosis model training device according to a third embodiment of the present application;

[0052] Figure 7 This is a structural block diagram of the auxiliary diagnosis model training device of the fourth embodiment of the present application. DETAILED DESCRIPTION

[0053] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0054] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0055] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0056] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0057] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0058] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0059] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0060] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0061] The first embodiment of the present application provides an auxiliary diagnosis model training method, which is applied to a client. Figure 1This is a flowchart of the auxiliary diagnosis model training method of the first embodiment of the present application, see Figure 1 As shown, the auxiliary diagnosis model training method of the first embodiment of the present application may specifically include the following steps.

[0062] S110, constructing a first training data set for training a first auxiliary diagnosis model.

[0063] The first auxiliary diagnosis model includes a first feature extraction network and a first classification network. The first feature extraction network is used to extract feature information from the input data of the first auxiliary diagnosis model. Optionally, the first feature extraction network includes but is not limited to a recurrent neural network (RNN), a convolutional neural network (CNN), a Transformer network, and the like. The first classification network outputs an auxiliary diagnosis result based on the feature information extracted by the first feature extraction network.

[0064] The first training data set includes medical images, labels corresponding to the medical images, public images selected from a public gallery that match the medical images, and labels corresponding to the public images. The medical images are private medical images of the client, and the medical images may include various detection images used by doctors for medical diagnosis. For example, X-ray images, electronic computed tomography (CT) images, magnetic resonance imaging images, and ultrasound detection images, etc. Optionally, the labels corresponding to the medical images may include labels manually added by medical experts, and may also include pseudo labels predicted by the first auxiliary diagnosis model. The labels are used to identify auxiliary diagnosis results. For example, the labels can be used to identify the auxiliary diagnosis results as healthy or sick, and can also be used to identify specific physical conditions, such as specific types of illness, etc.

[0065] The public gallery is a public image database that can be obtained through legal channels. The public gallery includes but is not limited to the MNIST database, the MS-COCO database, the ImageNet database, the Open Images Dtaset database, the VisualQA database, and the like. The public images are images selected from the public gallery that match the medical images. The public images may include various types of images that can be obtained from the public gallery. For example, images of people, images of animals, images of natural environments, images of vehicles, and images of characters, and the like. The labels of the public images may be the same as the labels of the matching medical images, or may indicate the same auxiliary diagnosis results as the labels of the matching medical images.

[0066] Optionally, the client may obtain medical data of the client. The medical data may include a first medical image and a label corresponding to the first medical image. A public image matching the first medical image is selected from a public gallery, and a correspondence between the public image and the label of the first medical image is constructed. That is, the label of the first medical image is assigned to the matching public image as the label of the public image. Then, based on the first medical image, the public image, and the labels corresponding to the first medical image and the public image, the first training data set is constructed.

[0067] Optionally, the medical images and public images in the first training data set may be original images of the medical images or the public images, or may be image data matrices generated based on the medical images or the public images. Optionally, the image size of the medical images and the image size of the public images may both be modified to target sizes. For example, the pixel sizes of the medical images and the public images may both be modified to 224*224. The medical images with the target size are converted into medical image data matrices, and the public images with the target size are converted into public image data matrices. Subsequently, the medical image data matrix and the public image data matrix may be used as samples of the first training data set.

[0068] Optionally, the first training data set may include a source domain and a target domain, the source domain may include public images and labels corresponding to the public images, and the target domain may include medical images and labels corresponding to the medical images.

[0069] S120: Input samples in the first training data set into the first auxiliary diagnosis model, and iteratively train the first auxiliary diagnosis model using a first optimization function, so that the first auxiliary diagnosis model meets a first optimization goal.

[0070] Wherein, the first optimization goal at least includes aligning the feature distribution of first feature information and the feature distribution of second feature information, the first feature information includes the feature information extracted from the medical image by the first feature extraction network, and the second feature information includes the feature information extracted from the public image by the first feature extraction network.

[0071] After the first training data set is constructed, the samples in the first training data set are input into the first auxiliary diagnosis model. For example, the medical images and public images in the first training data set can be used as input data of the first auxiliary diagnosis model, and the labels can be used as output data of the first auxiliary diagnosis model. The model parameters of the first auxiliary diagnosis model are optimized by the first optimization function, so that the feature distributions of the first feature information extracted from the medical image and the second feature information extracted from the public image by the first feature extraction network gradually tend to be aligned, such as Figure 2 shown.

[0072] It is understandable that the alignment of the feature distribution of the first feature information and the feature distribution of the second feature information described herein does not mean that the first feature information and the second feature information are completely identical. Rather, in the process of training the first auxiliary diagnosis model using the first training data set, the feature distribution difference between the first feature information and the second feature information is gradually reduced until the feature distribution difference between the first feature information and the second feature information changes very little or no longer reduces, and it can be determined that the feature distribution between the first feature information and the second feature information has been aligned, and further it can be determined that the training of the first auxiliary diagnosis model is completed.

[0073] In a specific implementation, the result of the first optimization function can be used to determine whether the feature distributions of the first feature information and the second feature information are aligned. For example, during the iterative training of the first auxiliary diagnosis model, if the result of the first optimization function changes very little or basically does not change between generations, it can be determined that the feature distributions of the first feature information and the second feature information are aligned. Of course, the training end condition of the first auxiliary diagnosis model can also be executed N times. That is, after performing N times of iterative training on the first auxiliary diagnosis model based on the first training data set, it can be determined that the training of the first auxiliary diagnosis model is completed.

[0074] S130, sending the first auxiliary diagnosis model and identification information for identifying the public image to the server, so that the server trains a second auxiliary diagnosis model based on the identification information and the first auxiliary diagnosis model, so that the second auxiliary diagnosis model meets the second optimization goal.

[0075] Optionally, after the training of the first auxiliary diagnosis model is completed, the first auxiliary diagnosis model as a whole may be saved as a file, or the model parameters of the first auxiliary diagnosis model may be saved as a file. Then, the file is sent to the server so that the server can run the first auxiliary diagnosis model based on the file.

[0076] Optionally, the identification information is used to identify a public image in the first training data set. The identification information includes but is not limited to an index value, name, category, etc. of the public image in the public gallery. The server obtains the identification information and can obtain a public image from the public gallery based on the identification information, and construct a second training data set based on the public image.

[0077] The second auxiliary diagnosis model includes a second feature extraction network and a second classification network. The second optimization goal at least includes aligning feature distributions between third feature information and fourth feature information, wherein the third feature information includes feature information extracted from the public image by the server using the first feature extraction network, and the fourth feature information includes feature information extracted from the public image by the server using the second feature extraction network.

[0078] Optionally, the server may train the first auxiliary diagnosis model based on the second training data set, and align the feature distribution of the third feature information and the fourth feature information on the second auxiliary diagnosis model through a second optimization function, thereby indirectly aligning the feature distribution between the feature information extracted from the public image by the second feature extraction network and the feature information extracted from the medical image by the first feature extraction network, so that the prediction results of the first auxiliary diagnosis model and the second auxiliary diagnosis model tend to be consistent, thereby achieving the purpose of training the second auxiliary diagnosis model without the server being exposed to the medical image. Figure 3 shown.

[0079] In the auxiliary diagnosis model training method of the embodiment of the present application, the first training data set includes not only the private medical images of the client, but also includes public images selected from the public gallery that match the medical images, and the labels of the medical images are assigned to the public images, and the medical images are supplemented by the public images to increase the sample size of the first training data set. In the process of training the first auxiliary diagnosis model in the client, the model parameters of the first auxiliary diagnosis model are optimized by the first optimization function, driving the feature distribution between the feature information extracted from the public images and the feature information extracted from the medical images by the first feature extraction network to align.

[0080] After the training of the first auxiliary diagnosis model is completed, the first auxiliary diagnosis model and the public image are sent to the server as a knowledge transfer medium. During the training of the second auxiliary diagnosis model by the server, the feature distribution between the feature information extracted from the public image by the second feature extraction network and the feature information extracted from the public image by the first feature extraction network is driven to align, thereby indirectly driving the feature distribution between the feature information extracted from the public image by the second feature extraction network and the feature information extracted from the medical image by the first feature extraction network to align, thereby achieving the purpose of making the prediction results of the second auxiliary diagnosis model and the first auxiliary diagnosis model tend to be consistent.

[0081] In this way, the requirements for the technical field correlation between the public images and the medical images are relatively low, allowing the use of public images that are significantly different from medical images to assist in the training of the first auxiliary diagnosis model and the second auxiliary diagnosis model, and being able to form better training effects, which is beneficial to reducing the difficulty of training large-size auxiliary diagnosis models.

[0082] Cooperate Figure 4 As shown, in some embodiments, step S110, constructing a first training data set for training a first auxiliary diagnosis model, may include the following steps.

[0083] S111, obtaining medical data of the client, wherein the medical data includes a first medical image, a second medical image, and a label corresponding to the first medical image.

[0084] S112, selecting a public image that matches the first medical image from a public image library, and constructing a corresponding relationship between the public image and the label of the first medical image.

[0085] S113, based on the first auxiliary diagnosis model, select a third medical image whose confidence meets the first threshold from the second medical image, and generate a label corresponding to the third medical image.

[0086] S114: construct the first training data set based on the first medical image, the third medical image, the public image and the label.

[0087] It can be understood that the label includes a label corresponding to the first medical image, a label corresponding to the third medical image, and a label corresponding to the public image. Optionally, the label corresponding to the first medical image may be an artificially marked label, and the first medical image can be understood as a labeled sample here. Correspondingly, the second medical image can be understood as an unlabeled sample. That is, the first medical image has an artificially marked auxiliary diagnosis result, and the second medical image does not have an artificially marked auxiliary diagnosis result. On this basis, the data set consisting of the first medical image and the label corresponding to the first medical image can be regarded as a labeled sample set. The data set consisting of the second medical images can be regarded as an unlabeled sample set

[0088] Optionally, after obtaining the private medical data of the client, a matching public image can be selected from the public image library based on the first medical image, and the label of the first medical image can be assigned to the public image as the label of the public image. The data set consisting of the public image and the label corresponding to the public image can be regarded as the source domain

[0089] For the second medical image as an unlabeled sample, the second medical image can be input into the first auxiliary diagnosis model, and the first auxiliary diagnosis model outputs the predicted probability of mapping the second medical image to each given category. The predicted probability can characterize the confidence of mapping the second medical image to each given category to a certain extent. The second medical image with a predicted probability greater than the first threshold can be selected as the third medical image, and the given category corresponding to the predicted probability can be used as the pseudo label of the third medical image. The third medical image is actually a high-confidence unlabeled sample.

[0090] In this way, training samples can be generated based on unlabeled medical images in the client, the number of medical images in the first training data set can be increased, and the training of the first training data set can be completed even when there are only a small number of labeled medical images and a large number of unlabeled medical images in the client. Moreover, inferior samples with low prediction probability can be filtered out to ensure the training effect of the first auxiliary diagnosis model.

[0091] Optionally, the third medical image may be selected based on the following formula:

[0092]

[0093] in, A pseudo label representing the third medical image; represents the unlabeled sample set, that is, the data set consisting of the second medical image. i express The i-th sample in . k represents the predicted label, that is, the given category. δ k Represents the predicted probability of the sample being mapped to the k-th category prediction label; represents the first auxiliary diagnosis model in the jth client; γ represents the first threshold.

[0094] Based on the above formula, samples with a predicted probability greater than the first threshold can be selected from the unlabeled sample set as the third medical image. Moreover, the given category with a predicted probability greater than the first threshold can be used as the pseudo label of the third medical image. The dataset composed of the first medical image and the third medical image can be regarded as the target domain Based on the source domain and the target domain Construct the first training dataset

[0095] Optionally, when the first medical image and the third medical image are acquired, the data set consisting of the first medical image and the third medical image can also be regarded as a high confidence sample set. For high confidence sample sets The samples in the high confidence sample set are subjected to data enhancement processing such as cropping, rotation, resizing, and recoloring to obtain enhanced samples. The samples in the high confidence sample set and the enhanced samples are considered as the target domain. In this way, not only the number of samples in the first training data set can be increased, but also the feature distributions of similar samples can be brought closer to each other.

[0096] In some embodiments, step S112, selecting a public image matching the first medical image from a public gallery, may include the following steps.

[0097] S1121, comparing a class of first medical images having the same label with various types of public images in the public image library to determine the similarity of data distribution between the class of first medical images and various types of public images.

[0098] S1122: Select a public image whose similarity meets a first preset condition as a public image that matches the first medical image of this type.

[0099] In this way, public images with data distribution similar to that of the first medical image can be selected from the public gallery. It is easier to achieve feature distribution alignment between the first medical image and the public image with similar data distribution, which is beneficial to improving the training effect of the first auxiliary diagnosis model and reducing the training difficulty of the first auxiliary diagnosis model.

[0100] Optionally, for a class of first medical images with the same label, a class of public images may be selected from the public image library. In this case, the first preset condition may be to select a class of public images with the highest similarity. For a class of first medical images with the same label, multiple classes of public images may also be selected from the public image library. In this case, the first preset condition may be to select multiple classes of public images with the highest similarity.

[0101] In specific implementation, the similarity of data distribution between the first medical image and the public image can be compared in a variety of ways. The following is a detailed description of comparing the similarity of data distribution between the first medical image and the public image in conjunction with a preferred implementation, but it should not be understood as being limited to comparing the similarity of data distribution between the first medical image and the public image in the following manner.

[0102] In some embodiments, step S1121, comparing a class of first medical images having the same label with various types of public images in the public gallery to determine the similarity of data distribution between the class of first medical images and various types of public images, may include the following steps.

[0103] B1, generating a first data matrix based on a class of first medical images with the same label.

[0104] Optionally, the image size of a class of first medical images with the same label can be adjusted to the target size to achieve the purpose of unifying the image size of the first medical images. For example, the pixel size of the first medical images can be uniformly adjusted to 28*28 or 224*224, etc. A medical data matrix can be generated based on each first medical image, and multiple medical data matrices can be spliced ​​to form the first data matrix X1. The number of images of this class of first medical images can be recorded as n1, and the target size can be recorded as d, then the first data matrix X1 is an n1×d matrix.

[0105] B2, generating a second data matrix based on a type of public images in the public database.

[0106] Optionally, similar to unifying the image size of the first medical image, the image size of the public image of this type may be adjusted to the target size to unify the image size of the public image of this type. Then, a public data matrix may be generated based on each public image, and multiple public data matrices may be spliced ​​into a second data matrix X2. The number of images of this type of public image may be recorded as n2, and the target size may be recorded as d, then the second data matrix X2 is an n2×d matrix.

[0107] B3, concatenate and transpose the first data matrix and the second data matrix to form a joint matrix.

[0108] Taking the first data matrix X1 as an n1×d matrix and the second data matrix X2 as an n2×d matrix as an example, the first data matrix X1 and the second data matrix X2 can be concatenated to form a (n1+n2)×d matrix. Then, the (n1+n2)×d matrix can be transposed to a d×(n1+n2) matrix, and the d×(n1+n2) matrix is ​​used as a joint matrix.

[0109] B4, performing non-negative matrix decomposition on the joint matrix to obtain a shared basis matrix and a coefficient matrix; wherein the coefficient matrix includes a first coefficient matrix and a second coefficient matrix, the first coefficient matrix and the second coefficient matrix can be spliced ​​horizontally to form the coefficient matrix, the first coefficient matrix represents the contribution degree of the shared basis matrix to the first data matrix, and the second coefficient matrix represents the contribution degree of the shared basis matrix to the second data matrix.

[0110] Optionally, a non-negative matrix factorization may be performed on the joint matrix based on a given rank r and an objective function as follows:

[0111] L_{jnmf}=φKL(X1|AS1)+KL(X2|AS2) (2).

[0112] Among them, L_{jnmf} represents the objective function; φ is the coefficient, which is |X2| / |X1|; X1 represents the first data matrix; X2 represents the second data matrix; |X1| represents the number of images that form the first data matrix. Taking the first data matrix n1×d matrix as an example, |X1|=n1; |X2| represents the number of images that form the second data matrix; KL represents relative entropy, that is, KL divergence; A represents the shared basis matrix, which is a d×r matrix; S1 represents the first coefficient matrix, which is an r×n1 matrix; S2 represents the second coefficient matrix, which is an r×n2 matrix.

[0113] On this basis, the shared base matrix A, the first coefficient matrix S1 and the second coefficient matrix S2 may have the following relationship:

[0114]

[0115]

[0116]

[0117] Wherein, T represents transformation. In specific implementation, the initial values ​​of A, S1 and S2 can be given, and the joint matrix can be iteratively decomposed based on formula (2) until the objective function is minimized, thereby obtaining the shared base matrix A, the first coefficient matrix S1 and the second coefficient matrix S2.

[0118] B5, based on the similarity between the first coefficient matrix and the second coefficient matrix during implementation, determine the similarity between the first medical image of this type and the public image of this type.

[0119] Since the first coefficient matrix represents the contribution degree of the shared base matrix to the first data matrix, and the second coefficient matrix represents the contribution degree of the shared base matrix to the second data matrix, the similarity between the first coefficient matrix and the second coefficient matrix can represent the similarity between the first data matrix and the second data matrix, and further can represent the similarity between the first medical image and the public image.

[0120] In order to determine the similarity between the first coefficient matrix and the second coefficient matrix, a similarity vector V with a length of r may be constructed. The number of elements r in the similarity vector V is the same as the number of rows in the first coefficient matrix S1 and the second coefficient matrix S2. λ Characterizes the similarity between the λth row in the first coefficient matrix S1 and the λth row in the second coefficient matrix S2.

[0121] A second threshold value Th may be preset for each row in the first coefficient matrix S1 and the second coefficient matrix S2. The specific value of the second threshold value Th may be a random number between zero and the maximum value of each row in the first coefficient matrix S1 and the second coefficient matrix S2. Then, v is calculated by counting the frequency of the element contribution value of the λth row of the first coefficient matrix S1 and the second coefficient matrix S2 exceeding the second threshold value Th. λ , the specific formula is as follows:

[0122]

[0123] in, represents the number of contribution values ​​of the λth row in the first coefficient matrix S1 exceeding the second threshold Th; It indicates the number of contribution values ​​of the λth row in the second coefficient matrix S2 exceeding the second threshold Th.

[0124] The similarity between the first coefficient matrix S1 and the second coefficient matrix S2, that is, the similarity between the first data matrix X1 and the second data matrix X2, can be determined by the following formula:

[0125]

[0126] in, Represents the average value of the elements in the similarity vector V; for The closer Dis(X1,X2) is to 0, the more similar the first data matrix X1 and the second data matrix X2 are, and further, the more similar the first medical image of this type is to the public image of this type.

[0127] In some embodiments, the first optimization function is used to optimize the parameters of the first auxiliary diagnosis model, align the feature distribution of the feature information extracted by the first feature extraction network from the public image and the first medical image, align the feature distribution of the feature information extracted by the first feature extraction network from similar samples in the target domain, align the category centroids of similar samples in the source domain and the target domain, so as to align the feature distribution of the feature information extracted by the first feature extraction network from similar samples in the source domain and the target domain, so that the first auxiliary diagnosis model meets the first optimization goal; wherein, the similar samples are samples with the same labels.

[0128] Optionally, the first optimization function is as follows:

[0129] L da =L ce +α*L h +β*L clu (8).

[0130] Among them, L da represents the first optimization function; L ce represents the supervised cross entropy loss function of samples in the source domain; l h represents the clustering loss function of samples in the target domain; l clu represents the centroid alignment loss function between samples in the source domain and samples in the target domain with the same label; α and β are hyperparameters. ce Under the correction, the feature distribution of the feature information extracted by the first feature extraction network from the public image and from the first medical image can gradually tend to be aligned.

[0131]

[0132]

[0133] in, represents the target domain in the first training dataset; represents the supervised contrast loss function of the i-th sample in the target domain; P(i) represents the set of samples in the target domain that have the same label as the i-th sample; represents the pseudo label of the i-th sample; represents the pseudo label of the αth sample; An index measuring the cosine similarity between two elements, τ represents the temperature scalar parameter; z i 、z α 、z β They respectively represent the feature information extracted by the first feature extraction network from the i-th sample, the α-th sample, and the β-th sample in the target domain.

[0134] In the first auxiliary diagnosis model training process, in L h Under the correction, the feature distribution of the feature information extracted by the first feature extraction network from the samples with the same label in the target domain can gradually tend to be aligned.

[0135] In order to narrow the cross-domain differences between samples in the source domain and samples in the target domain with the same label, the category centroid of the samples in the target domain and the category centroid of the samples in the source domain can be matched to align the feature distribution of the feature information extracted by the first feature extraction network from the samples in the source domain and the samples in the target domain with the same label.

[0136] Optionally, the class centroid of the sample can be calculated using the following formula:

[0137]

[0138] Where k represents the label; C k represents the class centroid of the sample with the k-th class label, which can be referred to as the centroid of the k-th class sample; δ k Represents the predicted probability of the sample being mapped to the k-th category prediction label; represents the first auxiliary diagnosis model of the jth client; express The first feature extraction network in ; Represents the feature information extracted from the sample by the first feature extraction network. Using the weighted average of the predicted probability and feature information as the class centroid of the k-th class sample, the predicted probability output by the first auxiliary diagnosis model can be used as a soft representation of the class centroid, which can enhance the generalization of the model.

[0139] Optionally, the cross-domain difference can be reduced by maximizing the similarity of the class centroids between samples in the source domain and samples in the target domain with the same label. On this basis, the centroid alignment loss function between samples in the source domain and samples in the target domain with the same label can be expressed by the following formula.

[0140]

[0141] in, Represents the class centroid of the k-th class sample in the target domain; represents the class centroid of the sample of the kth class in the source domain. In the training process of the first auxiliary diagnosis model, L clu Under the correction, the cross-domain differences between similar samples in the source domain and the target domain can be reduced, and the feature distribution of the feature information extracted by the first feature extraction network from similar samples in the source domain and the target domain can be aligned.

[0142] In some embodiments, the method may also include: obtaining the predicted probability of the public image based on the second auxiliary diagnosis model of the server; updating the label of the public image in the first training data set based on the predicted probability; and training the first auxiliary diagnosis model based on the updated first training data set. In this way, the knowledge of the second auxiliary model can be reversely transferred to the first auxiliary diagnosis model, and after multiple rounds of communication training, a well-generalized second auxiliary diagnosis model can be trained on the server.

[0143] A second embodiment of the present application provides an auxiliary diagnosis model training method, which is applied to a server. Figure 5 This is a flowchart of the auxiliary diagnosis model training method of the second embodiment of the present application, see Figure 5 As shown, the auxiliary diagnosis model training method of the second embodiment of the present application may specifically include the following steps.

[0144] S210, obtaining identification information and a first auxiliary diagnosis model trained by the client.

[0145] The first auxiliary diagnosis model is trained based on a first training data set, and the first training data set includes medical images of the client and public images selected from a public gallery that match the medical images. The first auxiliary diagnosis model includes a first feature extraction network and a first classification network, and the first auxiliary diagnosis model meets a first optimization goal, and the first optimization goal at least includes aligning the feature distribution of the first feature information and the feature distribution of the second feature information, the first feature information includes the feature information extracted from the medical image by the first feature extraction network, and the second feature information includes the feature information extracted from the public image by the first feature extraction network.

[0146] Optionally, after the training of the first auxiliary diagnosis model is completed, the first auxiliary diagnosis model as a whole may be saved as a file, or the model parameters of the first auxiliary diagnosis model may be saved as a file. Then, the file is sent to the server so that the server can run the first auxiliary diagnosis model based on the file.

[0147] The identification information is used to identify the public images in the first training data set. Optionally, the identification information is used to identify the public images in the first training data set. The identification information includes but is not limited to the index value, name, category, etc. of the public image in the public gallery.

[0148] S220: Construct a second training data set based on the identification information, where the second training data set includes at least part of the public images in the first training data set.

[0149] Optionally, the server obtains the identification information and can obtain the first training data set of the jth client from the public gallery based on the identification information. The public images selected are actually the first training dataset Source domain in Then, the second training data set can be constructed based on the public images

[0150] Optional, to filter out source domains The low-quality samples in the source domain can be input into the first auxiliary diagnosis model to obtain the prediction probability. Based on formula (1), the public images with prediction probabilities lower than the third threshold are filtered out, and the public images with prediction probabilities higher than the third threshold are retained. The public images with prediction probabilities higher than the third threshold can be recorded as

[0151] Optionally, in the case where multiple clients jointly train the second auxiliary diagnosis model of the server, the server may construct the second training data set based on the public images in the first training data set of the multiple clients. J represents the number of clients participating in joint training.

[0152] S230: Input the samples in the second training data set into a second auxiliary diagnosis model, and iteratively train the second auxiliary diagnosis model through a second optimization function, so that the second auxiliary diagnosis model meets the second optimization goal.

[0153] The second auxiliary diagnosis model f l The second feature extraction network and the second classification network may be included. The second optimization objective at least includes aligning the feature distribution of the third feature information and the feature distribution of the fourth feature information. The third feature information includes feature information extracted from the public images of the second training data set based on the first feature extraction network. The third feature information may be recorded as The fourth feature information includes feature information extracted from the public images in the second training data set based on the second feature extraction network. The fourth feature information can be recorded as

[0154] Optionally, the second optimization function may be configured to gradually reduce the third feature information by optimizing the model parameters of the second auxiliary diagnosis model. and the fourth characteristic information The relative entropy (also known as KL divergence) between them is used to align the third feature information and the fourth characteristic information The characteristic distribution between Figure 3 shown.

[0155] Optionally, the second optimization function can be expressed by the following formula:

[0156]

[0157] Among them, L server represents the second optimization function; E represents expected calculation, and KL represents relative entropy, that is, KL divergence.

[0158] Optionally, the method may further include: sending the predicted probability of the second auxiliary diagnosis model for the public image to the client, so that the client updates the label of the public image in the first training data set, and trains the first auxiliary diagnosis model based on the updated first training data set. In this way, the knowledge of the second auxiliary model can be reversely transferred to the first auxiliary diagnosis model. After multiple rounds of communication training, a well-generalized second auxiliary diagnosis model can be trained on the server. For example, the second auxiliary diagnosis model can be used for The average predicted probability of samples of the same type in the first training dataset is fed back to the client as the soft label of the samples of this type in the first training dataset.

[0159] See also Figure 6 As shown, the third embodiment of the present application provides an auxiliary diagnosis model training device, comprising:

[0160] The first construction module 301 is configured to construct a first training data set for training a first auxiliary diagnosis model; the first auxiliary diagnosis model includes a first feature extraction network and a first classification network; the first training data set includes a medical image, a public image selected from a public gallery and matching the medical image, and labels corresponding to the medical image and the public image, the labels being used to identify auxiliary diagnosis results;

[0161] A first training module 302 is configured to input samples in the first training data set into the first auxiliary diagnosis model, and iteratively train the first auxiliary diagnosis model through a first optimization function, so that the first auxiliary diagnosis model meets a first optimization goal; the first optimization goal at least includes aligning feature distributions of first feature information and second feature information, the first feature information includes feature information extracted from the medical image by the first feature extraction network, and the second feature information includes feature information extracted from the public image by the first feature extraction network;

[0162] The first communication module 303 is configured to send the first auxiliary diagnosis model and identification information for identifying the public image to the server, so that the server trains a second auxiliary diagnosis model based on the identification information and the first auxiliary diagnosis model, so that the second auxiliary diagnosis model meets the second optimization goal; the identification information is used by the server to construct a second training data set based on the public image; the second auxiliary diagnosis model includes a second feature extraction network and a second classification network; the second optimization goal at least includes aligning the feature distribution of third feature information and fourth feature information, the third feature information includes the feature information extracted by the server from the public image based on the first feature extraction network, and the fourth feature information includes the feature information extracted by the server from the public image based on the second feature extraction network.

[0163] In some embodiments, the first building module 301 is specifically configured as follows:

[0164] Acquire medical data of the client, where the medical data includes a first medical image and a label corresponding to the first medical image;

[0165] Selecting a public image matching the first medical image from a public image library, and establishing a corresponding relationship between the public image and the label of the first medical image;

[0166] The first training data set is constructed based on the first medical image, the public image and the label.

[0167] In some embodiments, the first building module 301 is specifically configured as follows:

[0168] Acquire medical data of the client, the medical data including a first medical image, a second medical image, and a label corresponding to the first medical image;

[0169] Based on the first auxiliary diagnosis model, selecting a third medical image whose confidence meets a first threshold from the second medical image, and generating a label corresponding to the third medical image;

[0170] Selecting a public image matching the first medical image from a public image library, and establishing a corresponding relationship between the public image and the label of the first medical image;

[0171] The first training data set is constructed based on the first medical image, the third medical image, the public image and the label.

[0172] In some embodiments, the first building module 301 is further configured to:

[0173] Perform data enhancement processing on the first medical image and / or the third medical image to increase the number of samples in the first training data set.

[0174] In some embodiments, the first building module 301 is specifically configured as follows:

[0175] Comparing a class of first medical images with the same label with various types of public images in the public image library to determine the similarity of data distribution between the class of first medical images and various types of public images;

[0176] The public image whose similarity meets the first preset condition is selected as the public image that matches the first medical image of this type.

[0177] In some embodiments, the first building module 301 is specifically configured as follows:

[0178] Based on a class of first medical images having the same label, generating a first data matrix;

[0179] Based on a type of public images in the public database, generating a second data matrix;

[0180] Concatenating and transposing the first data matrix and the second data matrix to form a joint matrix;

[0181] Performing non-negative matrix decomposition on the joint matrix to obtain a shared basis matrix and a coefficient matrix; wherein the coefficient matrix includes a first coefficient matrix and a second coefficient matrix, the first coefficient matrix and the second coefficient matrix are horizontally spliced ​​to form the coefficient matrix, the first coefficient matrix represents the contribution degree of the shared basis matrix to the first data matrix, and the second coefficient matrix represents the contribution degree of the shared basis matrix to the second data matrix;

[0182] Based on the similarity between the first coefficient matrix and the second coefficient matrix, the similarity between the first medical image of the type and the public image of the type is determined.

[0183] In some embodiments, the first training data set includes a source domain and a target domain, the source domain includes the public image and a label corresponding to the public image, and the target domain includes the first medical image, the third medical image, and labels corresponding to the first medical image and the third medical image;

[0184] The first optimization function is used to optimize the parameters of the first auxiliary diagnosis model, align the feature distribution of the feature information extracted by the first feature extraction network from the public image and the first medical image, align the feature distribution of the feature information extracted by the first feature extraction network from similar samples in the target domain, align the category centroids of similar samples in the source domain and the target domain, so as to align the feature distribution of the feature information extracted by the first feature extraction network from similar samples in the source domain and the target domain, so that the first auxiliary diagnosis model meets the first optimization goal; wherein, the similar samples are samples with the same labels.

[0185] In some embodiments, the first communication module is further configured to: obtain the predicted probability of the server for the public image based on the second auxiliary diagnosis model;

[0186] The first construction module 301 is further configured to: update the labels of the public images in the first training data set based on the predicted probability;

[0187] The first training module is further configured to: train the first auxiliary diagnosis model based on the updated first training data set.

[0188] See also Figure 7 As shown, the fourth embodiment of the present application provides an auxiliary diagnosis model training device, comprising:

[0189] The second communication module 401 is configured to obtain identification information and a first auxiliary diagnosis model trained by a client; the first auxiliary diagnosis model is trained based on a first training data set, the first training data set includes medical images of the client and public images selected from a public gallery and matching the medical images; the identification information is used to identify the public images in the first training data set; the first auxiliary diagnosis model includes a first feature extraction network and a first classification network, the first auxiliary diagnosis model meets a first optimization goal, the first optimization goal at least includes aligning feature distributions of first feature information and second feature information, the first feature information includes feature information extracted from the medical image by the first feature extraction network, and the second feature information includes feature information extracted from the public image by the first feature extraction network;

[0190] A second construction module 402 is configured to construct a second training data set based on the identification information, wherein the second training data set includes at least part of the public images in the first training data set;

[0191] The second training module 403 is configured to input samples in the second training data set into a second auxiliary diagnosis model, and iteratively train the second auxiliary diagnosis model through a second optimization function so that the second auxiliary diagnosis model meets the second optimization goal; the second auxiliary diagnosis model includes a second feature extraction network and a second classification network; the second optimization goal at least includes aligning the feature distribution of the third feature information and the feature distribution of the fourth feature information, the third feature information includes feature information extracted from the public images in the second training data set based on the first feature extraction network, and the fourth feature information includes feature information extracted from the public images in the second training data set based on the second feature extraction network.

[0192] An embodiment of the present application further provides an electronic device, comprising at least a memory and a processor, wherein the memory stores an application program, and the processor implements the method described in any of the above embodiments when executing the application program on the memory.

[0193] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions in the computer-readable storage medium are executed, the method described in any of the above embodiments is implemented.

[0194] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, electronic equipment, computer-readable storage media or computer program products. Therefore, the present application may adopt the form of complete hardware embodiments, complete software embodiments or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media that include computer-usable program code. When implemented by software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0195] The processor may be a general purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. The general purpose processor may be a microprocessor or any conventional processor.

[0196] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0197] The above-mentioned readable storage medium can be a disk, an optical disk, a DVD, a USB, a read-only memory (ROM) or a random access memory (RAM), etc. This application does not limit the specific form of the storage medium.

[0198] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A method for training an auxiliary diagnosis model, characterized in that: Applied to the client, including: Constructing a first training data set for training a first auxiliary diagnosis model; the first auxiliary diagnosis model includes a first feature extraction network and a first classification network; the first training data set includes a medical image of a client, a label corresponding to the medical image, a public image selected from a public gallery that matches the medical image, and a label corresponding to the public image, wherein the label is used to identify an auxiliary diagnosis result; Input samples in the first training data set into the first auxiliary diagnosis model, and iteratively train the first auxiliary diagnosis model through a first optimization function, so that the first auxiliary diagnosis model meets a first optimization goal; the first optimization goal at least includes aligning feature distributions of first feature information and second feature information, the first feature information includes feature information extracted from the medical image by the first feature extraction network, and the second feature information includes feature information extracted from the public image by the first feature extraction network; The first auxiliary diagnosis model and identification information for identifying the public image are sent to the server, so that the server trains a second auxiliary diagnosis model based on the identification information and the first auxiliary diagnosis model, so that the second auxiliary diagnosis model meets the second optimization goal; the identification information is used by the server to obtain the public image and construct a second training data set; the second auxiliary diagnosis model includes a second feature extraction network and a second classification network; the second optimization goal at least includes aligning the feature distribution of third feature information and fourth feature information, the third feature information includes the feature information extracted by the server from the public image based on the first feature extraction network, and the fourth feature information includes the feature information extracted by the server from the public image based on the second feature extraction network.

2. The method according to claim 1, characterized in that The step of constructing a first training data set for training a first auxiliary diagnosis model comprises: Acquire medical data of the client, where the medical data includes a first medical image and a label corresponding to the first medical image; Selecting a public image matching the first medical image from a public image library, and establishing a corresponding relationship between the public image and the label of the first medical image; The first training data set is constructed based on the first medical image, the public image and the label.

3. The method according to claim 1, characterized in that The step of constructing a first training data set for training a first auxiliary diagnosis model comprises: Acquire medical data of the client, the medical data including a first medical image, a second medical image, and a label corresponding to the first medical image; Based on the first auxiliary diagnosis model, selecting a third medical image whose confidence meets a first threshold from the second medical image, and generating a label corresponding to the third medical image; Selecting a public image matching the first medical image from a public image library, and establishing a corresponding relationship between the public image and the label of the first medical image; The first training data set is constructed based on the first medical image, the third medical image, the public image and the label.

4. The method according to claim 3, characterized in that The constructing of a first training data set for training a first auxiliary diagnosis model further includes: Perform data enhancement processing on the first medical image and / or the third medical image to increase the number of samples in the first training data set.

5. The method according to claim 2 or 3, characterized in that: The selecting a public image matching the first medical image from a public image library comprises: Comparing a class of first medical images with the same label with various types of public images in the public image library to determine the similarity of data distribution between the class of first medical images and various types of public images; The public image whose similarity meets the first preset condition is selected as the public image that matches the first medical image of this type.

6. The method according to claim 5, characterized in that The step of comparing a class of first medical images having the same label with various types of public images in the public image library to determine the similarity of data distribution between the class of first medical images and various types of public images includes: Based on a class of first medical images having the same label, generating a first data matrix; Based on a type of public images in the public database, generating a second data matrix; Concatenating and transposing the first data matrix and the second data matrix to form a joint matrix; Performing non-negative matrix decomposition on the joint matrix to obtain a shared basis matrix and a coefficient matrix; wherein the coefficient matrix includes a first coefficient matrix and a second coefficient matrix, the first coefficient matrix and the second coefficient matrix are horizontally spliced ​​to form the coefficient matrix, the first coefficient matrix represents the contribution degree of the shared basis matrix to the first data matrix, and the second coefficient matrix represents the contribution degree of the shared basis matrix to the second data matrix; Based on the similarity between the first coefficient matrix and the second coefficient matrix, the similarity between the first medical image of the type and the public image of the type is determined.

7. The method according to claim 3, characterized in that The first training data set includes a source domain and a target domain, the source domain includes the public image and a label corresponding to the public image, and the target domain includes the first medical image, the third medical image, and the labels corresponding to the first medical image and the third medical image; The first optimization function is used to optimize the parameters of the first auxiliary diagnosis model, align the feature distribution of the feature information extracted by the first feature extraction network from the public image and the first medical image, align the feature distribution of the feature information extracted by the first feature extraction network from similar samples in the target domain, align the category centroids of similar samples in the source domain and the target domain, so as to align the feature distribution of the feature information extracted by the first feature extraction network from similar samples in the source domain and the target domain, so that the first auxiliary diagnosis model meets the first optimization goal; wherein, the similar samples are samples with the same labels.

8. The method according to claim 3, characterized in that The method further comprises: Obtaining the predicted probability of the public image by the server based on the second auxiliary diagnosis model; Based on the predicted probability, updating the labels of the public images in the first training data set; The first auxiliary diagnosis model is trained based on the updated first training data set.

9. A method for training an auxiliary diagnosis model, characterized in that: Applied to the server, including: Obtain identification information and a first auxiliary diagnosis model trained by a client; the first auxiliary diagnosis model is trained based on a first training data set, the first training data set includes medical images of the client and public images selected from a public gallery and matching the medical images; the identification information is used to identify the public images in the first training data set; the first auxiliary diagnosis model includes a first feature extraction network and a first classification network, the first auxiliary diagnosis model meets a first optimization goal, the first optimization goal at least includes aligning feature distributions of first feature information and second feature information, the first feature information includes feature information extracted from the medical image by the first feature extraction network, and the second feature information includes feature information extracted from the public image by the first feature extraction network; Based on the identification information, construct a second training data set, where the second training data set includes at least part of the public images in the first training data set; The samples in the second training data set are input into a second auxiliary diagnosis model, and the second auxiliary diagnosis model is iteratively trained through a second optimization function, so that the second auxiliary diagnosis model meets the second optimization goal; the second auxiliary diagnosis model includes a second feature extraction network and a second classification network; the second optimization goal at least includes aligning the feature distribution of the third feature information and the feature distribution of the fourth feature information, the third feature information includes feature information extracted from the public images of the second training data set based on the first feature extraction network, and the fourth feature information includes feature information extracted from the public images of the second training data set based on the second feature extraction network.

10. An auxiliary diagnosis model training device, characterized in that: include: A first construction module is configured to construct a first training data set for training a first auxiliary diagnosis model; The first auxiliary diagnosis model includes a first feature extraction network and a first classification network; The first training data set includes a medical image of a client, a label corresponding to the medical image, a public image selected from a public image library that matches the medical image, and a label corresponding to the public image, wherein the label is used to identify an auxiliary diagnosis result; a first training module, configured to input samples in the first training data set into the first auxiliary diagnosis model, and iteratively train the first auxiliary diagnosis model through a first optimization function, so that the first auxiliary diagnosis model meets a first optimization goal; the first optimization goal at least includes aligning feature distributions of first feature information and second feature information, the first feature information includes feature information extracted from the medical image by the first feature extraction network, and the second feature information includes feature information extracted from the public image by the first feature extraction network; The first communication module is configured to send the first auxiliary diagnosis model and identification information for identifying the public image to the server, so that the server trains a second auxiliary diagnosis model based on the identification information and the first auxiliary diagnosis model, so that the second auxiliary diagnosis model meets the second optimization goal; the identification information is used by the server to obtain the public image and construct a second training data set; the second auxiliary diagnosis model includes a second feature extraction network and a second classification network; the second optimization goal at least includes aligning the feature distribution of third feature information and fourth feature information, the third feature information includes feature information extracted by the server from the public image based on the first feature extraction network, and the fourth feature information includes feature information extracted by the server from the public image based on the second feature extraction network.