A method for training a multi-disease referral system, a multi-disease referral system, and a method

Abstract

The present invention provides a method for constructing a multi-disease referral system, including: S1. Obtaining training samples of fundus images for training, and labeling the fundus images with all diseases corresponding to each sample as positive labels to obtain a training data set; S2. Inputting the training samples in the training data set into the multi-disease referral system to be trained, wherein the multi-disease referral system includes multiple disease branch networks, and each disease branch network includes a classifier corresponding to a disease and an activation layer for processing the output of the classifier; the multi-disease referral system further includes a normalization layer for calculating the outputs of the classifiers of all disease branch networks; S3. Updating the weights of the multiple disease branch networks according to the loss function of the training samples until convergence; wherein the loss function includes the losses of all disease branch networks plus the multi-label regression loss of the merged outputs of the classifiers of all disease branch networks.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence. Specifically, it relates to the application of artificial intelligence in the medical field. More specifically, it relates to a method for training a multi-disease referral system, a multi-disease referral system, and a multi-disease referral method. Background Art

[0002] Fundus images contain rich vascular health information. Through the lesion characteristics appearing in fundus images, various diseases can be diagnosed or assisted in diagnosis. As a classic model of deep learning, a convolutional neural network can learn parameters from a large amount of fundus image data to obtain powerful feature extraction capabilities and realize the judgment of fundus image lesions. Based on the CNN model, a multi-disease referral system can be constructed, that is, inputting fundus images into the model for the judgment of multiple diseases. When the model judges that a certain disease's lesion characteristics appear in the fundus image, a referral is made for further diagnosis; if the model judges that the fundus image is normal without disease, no referral is made. When this system is actually applied in the actual medical scenario, it is necessary to ensure the accuracy of referrals and non-referrals, and at the same time, ensure a certain degree of discrimination between multiple diseases that require referrals.

[0003] For example, assume that the multi-disease referral system needs to judge whether an individual's fundus image is a normal healthy fundus or a referral case with certain diseases among diseases ①, ②, ③, and ④. Under the existing technology, the common practices are as follows:

[0004] One approach is to take these five as five categories, that is, directly classify the samples as healthy fundus having disease ①, having disease ②, having disease ③, having disease ④, and use the five-class SoftmaxLoss for training when training the model. However, this approach does not consider that there is no mutually exclusive relationship between different diseases, and some diseases even have a relatively high probability of co-occurrence. There are also samples with multiple disease labels in the training set. If it is simply regarded as a five-classification task, different diseases are regarded as mutually exclusive relationships, and the situation of co-occurring disease referrals cannot be handled. Therefore, this task is not suitable for being processed as a mutually exclusive classification task.

[0005] Another approach is to regard the judgment of each disease as a binary classification task. In the model training stage, four Sigmoid classifications + binary cross-entropy loss are used. Each classifier corresponds to one disease, and the model structure is as Figure 1 shown. In the application stage, judgments are made for each disease respectively according to the Sigmoid output and the threshold. The disadvantage of this approach is that it cannot guarantee the discrimination ability of the model for different diseases. In the training stage, each disease classification only trains the positive samples of that disease and healthy fundus (for example, classifier ①, which only trains the positive samples of positive sample ① and negative samples If there is a loss, the scores output by the classifier for samples positive for other diseases are not restricted. Due to the significant differences between diseased fundus and normal fundus, the output distribution of the classifier will be as Figure 2 shown Figure 1 In [Figure], classifier ① can better distinguish healthy samples and samples positive for disease ①, but will misclassify a large number of samples positive for other diseases as positive for disease ①. Similar situations exist for classifiers ②, ③, and ④, and the Sigmoid outputs of different classifiers are not comparable, that is, the model cannot distinguish between different diseases being positive, resulting in confusion in the referral reasons finally given by the system.

[0006] The third method is an improvement based on the second method. When training each classifier, train the classifier for samples positive for the corresponding disease and all samples negative for that disease, including healthy samples and samples positive for other diseases but negative for that disease (for example, for classifier ①, there is a loss for all other data of positive samples ① and negative samples). The following problems exist in this method: 1. Incomplete annotation. When annotating samples, the annotator is likely to only notice the most significant or most severe lesions presented in the image for some cases of concurrent diseases, resulting in missed annotation of secondary lesions and early lesions; 2. Feature overfitting. Since the features presented by the lesions of some diseases in the image are similar, violating this objective similarity and requiring the classifier for a certain disease to classify samples of other diseases with similar image features and healthy samples as a class of negative samples may cause the classifier to overfit to the unique features of the samples of that disease in the training set, and misclassify some samples with insignificant unique features as negative during application, resulting in FN (FN refers to false negative, that is, judging a positive as negative), affecting the accuracy of whether to refer. Summary of the Invention

[0007] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art, and provide a model training method for a multi-disease referral system, which can improve the discrimination of different diseases while ensuring the accuracy of judging whether to refer, and give more accurate referral reasons.

[0008] According to a first aspect of the present invention, there is provided a method for constructing a multi-disease referral system, the method comprising: S1, obtaining training samples of fundus images for training, and labeling the fundus images with all diseases corresponding to each sample as positive labels to obtain a training dataset; S2, inputting the training samples in the training dataset into the multi-disease referral system to be trained, wherein the multi-disease referral system includes a plurality of disease branch networks, each disease branch network includes a classifier corresponding to a disease and an activation layer for processing the output of the classifier; the multi-disease referral system further includes a normalization layer for calculating the outputs of the classifiers of all disease branch networks; S3, updating the weights of the plurality of disease branch networks according to the loss function of the training samples until convergence; wherein the loss function includes the losses of all disease branch networks plus the multi-label regression loss of the merged outputs of the classifiers of all disease branch networks.

[0009] In some embodiments of the present invention, the multi-disease referral system further includes a backbone convolutional neural network, and its output is respectively connected to each disease branch network.

[0010] Preferably, the activation layer of each disease branch network is a sigmoid layer, and the normalization layer is a Softmax layer.

[0011] In some embodiments of the present invention, the loss function is expressed as:

[0012] L = ∑L1 + L2

[0013] Wherein, L represents the total loss of the multi-disease referral system, ∑L1 is the sum of the losses of each disease branch network, each L1 is the loss generated by the samples containing the positive labels of the diseases corresponding to the current disease branch network and healthy samples, and L2 is the multi-label regression loss of the merged outputs of the classifiers of all disease branch networks. In some embodiments of the present invention, the loss of each disease branch network is a binary cross-entropy loss, expressed as:

[0014]

[0015]

[0016] Wherein, x represents the sample fundus image input into the disease branch network, and f i (x) represents the predicted output of x in the disease branch network i.

[0017] In some embodiments of the present invention, the merged multi-label regression loss is expressed as:

[0018]

[0019] Among them, x represents the sample fundus image input into the disease branch network, and f i (x) represents the output of the disease branch network i, and P represents the set of all positive labels included in the sample x.

[0020] In some embodiments of the present invention, the combined multi-label regression loss is expressed as:

[0021]

[0022] Among them, x represents the sample fundus image input into the disease branch network, and f i (x) represents the predicted output of x in the disease branch network i, and P represents the set of all positive labels included in the sample x.

[0023] According to a second aspect of the present invention, there is provided a multi-disease referral system trained by using the method described in the first aspect of the present invention. The system includes: a plurality of disease branch networks, where each disease branch network includes a classifier for a corresponding disease and an activation layer for processing the output of the classifier, and each disease branch network outputs its corresponding score and whether referral is required according to the input fundus image; a normalization layer for calculating the output of the classifiers of all disease branch networks to obtain the probability of the target object suffering from each disease and sorting the probabilities according to the disease severity, where the score output by the disease branch network is used as the basis for sorting the disease severity.

[0024] In some embodiments of the present invention, the activation layer of each disease branch network outputs the score of the input fundus image in the current disease branch network and judges whether referral is required based on a preset threshold; among them, the result of whether referral is required is obtained by comparing the score output by the current disease branch network with the preset threshold of the disease branch network, and it is determined that referral is required when the score of the input fundus image is greater than or equal to the preset threshold of the disease branch network.

[0025] Preferably, the preset threshold of each disease branch network is set in the following manner: T1. Obtain a test set with the same distribution as the application scenario, and use the constructed multi-disease referral system to give the score of each sample in the test set in the current disease branch network; T2. Calculate the TPR and FPR corresponding to all rate scores with the score of each sample as the to-be-set threshold, where TPR represents the true positive rate corresponding to the test set, and FPR represents the false positive rate corresponding to the test set; T3. Set the preset threshold of the current disease branch network to the to-be-set threshold that maximizes TPR + 1 - FPR.

[0026] According to the third aspect of the present invention, there is provided a multi-disease referral method, the method comprising: P1, obtaining a fundus image of a patient; P2, using the system described in the second aspect of the present invention to determine whether the patient needs to be referred and the disease probabilities sorted by disease significance; P3, referring the patient according to the disease probabilities sorted by disease significance.

[0027] Compared with the prior art, the advantages of the present invention are as follows: The present invention can be compatible with the situation of multiple concurrent diseases, accurately give the referral diseases and referral reasons, and ensure the distinction between different diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The following further describes embodiments of the present invention with reference to the accompanying drawings, wherein:

[0029] Figure 1 It is a schematic structural diagram of a binary disease referral system under the prior art according to an embodiment of the present invention;

[0030] Figure 2 It is a schematic diagram of the sample recognition and classification effect of a binary disease referral system under the prior art according to an embodiment of the present invention;

[0031] Figure 3 It is a schematic diagram of a multi-disease referral system according to an embodiment of the present invention;

[0032] Figure 4 It is a schematic diagram of the comparison relationship between a smoothing function and a maximum value function according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below through specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0034] First, the model structure used in the present invention will be introduced. To better understand the present invention, the present invention uses the example described in the background art, where the fundus image of the target object individual is a normal healthy fundus or a case to be referred with some of the four diseases ①, ②, ③, and ④. As described in the background art, it is necessary to consider the concurrency of diseases to ensure the accuracy of referral. For example Figure 3As shown in the figure, the multi-disease referral system of the present invention is in the form of a main convolutional neural network + four disease branch networks. The main convolutional neural network (main CNN) includes several convolutional layers, which are used as a shared main network to process the same parts of the four disease branch networks to reduce the complexity of the network structure. The four disease branch networks are disease branch network 1, disease branch network 2, disease branch network 3, and disease branch network 4 respectively. After each disease branch network, there are several convolutional layers for extracting corresponding disease features, a classifier for the corresponding disease, and an activation layer (the activation layer uses a sigmoid layer) for processing the output of the classifier. In this embodiment, disease branch network 1 corresponds to convolution 1, classifier 1, and sigmoid layer 1. The feature 1 extracted by convolution 1 is output to sigmoid layer 1 after passing through classifier 1, and then the score corresponding to the sample input into disease branch network 1 is output. Similarly, disease branch network 2 corresponds to convolution 2, classifier 2, and sigmoid layer 2. The feature 2 extracted by convolution 2 is output to sigmoid layer 2 after passing through classifier 2, and then the score corresponding to the sample input into disease branch network 2 is output. Disease branch network 3 corresponds to convolution 3, classifier 3, and sigmoid layer 3. The feature 3 extracted by convolution 3 is output to sigmoid layer 3 after passing through classifier 3, and then the score corresponding to the sample input into disease branch network 3 is output. Disease branch network 4 corresponds to convolution 4, classifier 4, and sigmoid layer 4. The feature 4 extracted by convolution 4 is output to sigmoid layer 2 after passing through classifier 4, and then the score corresponding to the sample input into disease branch network 4 is output. The outputs of the classifiers of all disease branch networks are connected to a Softmax layer. For those of ordinary skill in the art, it is obvious that the present invention is not limited to the above four disease branch networks, and there can be more or fewer than four disease branch networks to handle the corresponding number of disease referrals.

[0035] After the fundus image is input into the system and processed by the shared main CNN structure, it is respectively input into the four disease branch networks. The output of the classifier of the disease branch network is mapped to the score (0, 1) of the input fundus image in the corresponding disease branch network through sigmoid activation, and then it is judged whether referral is needed according to the threshold. When multiple branches judge that referral is needed, the Softmax layer calculates the probability of suffering from each disease according to the outputs of the classifiers of all disease branch networks and sorts them according to the disease significance. Among them, the score output after sigmoid processing is the basis for the disease significance ranking. The higher the score, the more prominent the disease significance.

[0036] The training process of the present invention will be described in detail below.

[0037] Since the specific neural network training process is an existing method, it will not be elaborated here. Only the processing of the training data set, the design of the loss function, and the setting of the threshold will be described below.

[0038] First, obtain training samples of fundus images for training, and label the fundus images with all diseases corresponding to each sample as positive labels to obtain a training dataset; among them, the number of positive labels to be marked for a fundus image is equal to the number of diseases it corresponds to. This method can avoid incomplete manual annotation and neglect of insignificant lesion features.

[0039] Then, use the fundus images marked with positive labels and healthy fundus images as the training dataset to train the disease branch networks respectively. At this time, in order to make the outputs of each branch of the above system network structure have distinctiveness and comparability for different diseases, and at the same time prevent overfitting, the inventor has made a special design for the loss function in training the multi-disease referral system of the present invention. The total loss function consists of two parts L = ∑L1 + L2. The first part ∑L1 is the training sample loss, which is the sum of the losses of all disease branch networks. The second part L2 is the regression loss between all disease branch networks added to combat the risk of overfitting and ensure the referral accuracy. According to an embodiment of the present invention, the first part of the loss adopts binary cross-entropy loss:

[0040]

[0041]

[0042] Among them, x represents the sample fundus image input to the disease branch network, and f i (x) represents the predicted output of x in the disease branch network i. This part of the loss function is only generated by healthy fundus and the positive samples of the diseases corresponding to each branch, and other disease positive samples do not generate losses. The sum of the binary cross-entropy losses of all disease branches constitutes the first part of the total loss function.

[0043] Taking the disease branch network 1 as an example, the first part of the loss can be expressed as:

[0044]

[0045]

[0046] f1(x) represents the output of the disease branch 1. Other disease positive samples without the ① label do not generate losses, and this sample also does not contribute to the L1 of branches 2-4. This part of the loss ensures the clear distinction of the model between healthy fundus and fundus that needs to be referred, without the risk of overfitting, and can ensure the accuracy of whether to refer.

[0047] To ensure the distinctiveness and comparability of diseases, as well as the identification of disease co-occurrences, when a sample has a positive label for a certain disease, it should be considered that the pathological features of this disease are the most significant or severe features presented by the sample. Therefore, the output of the corresponding branch of this disease needs to be greater than the outputs of other branches, and the co-occurrence of diseases should be fully considered. According to an embodiment of the present invention, the second part of the loss function is designed as a multi-label regression loss that combines the outputs of the classifiers of all disease branch networks:

[0048]

[0049] where \(P\) is the set of all positive labels of the sample, represents the maximum value in the outputs of the branch networks without the corresponding positive label for the sample. For example, assume a sample \(x\) contains two positive labels ① and ②, then \(P\) is the set containing ① and ②, is the maximum value in the outputs of the disease branch networks 3 and 4 for \(x\). Similarly, \(\min\) i∈P \(f\) i (\(x\)) represents the minimum value in the outputs of the branch networks 1 and 2 for \(x\). Through the second part of the loss function, the distinctiveness of the system for different diseases can be effectively increased, and the comparability between the outputs of the branch networks can be increased.

[0050] Taking the sample with only a single positive label ① as an example, the second part of the loss is:

[0051] \(L_2=\max(0,\max\) i≠1 \(f\) i (\(x\)) - \(f_1(x))\)

[0052] According to an embodiment of the present invention, a smooth function \(y = \log(1 + \exp(x))\) can be used to replace the maximum value function. As shown in Figure 4 the schematic diagram of the comparison relationship between the smooth function and the maximum value function, thus, the second part of the loss function can become:

[0053]

[0054] Taking the sample with only a single positive label ① as an example, the second part of the loss is:

[0055]

[0056] By adding the second part of the loss function, the loss function is optimized, which is similar to performing Softmax regression on the outputs of multiple branches. Therefore, the outputs of these different branches become comparable. The difference from the pure binary classification model in the prior art is that the loss function here does not require mutually exclusive sample labels and can be compatible with samples with multiple disease labels with co-occurrences.

[0057] As can be seen from the description of the above embodiments, the complete loss function of each disease branch network in the multi-disease referral system of the present invention includes the binary cross-entropy loss of each branch plus the combined multi-label SoftmaxLoss (regression loss). The former ensures the recall of each disease, thereby ensuring the overall referral accuracy of the model, and the latter ensures the differentiation of the model for different disease types. After inputting the fundus image into the trained model, if the result output by any branch exceeds the selected threshold after passing through the Sigmoid layer, it is determined that referral is required. Then, the outputs of the classifiers of all disease branch networks are calculated through the Softmax layer to obtain the proportion or ranking of each disease type in the referral reasons, and the score output after sigmoid is used as the basis for the disease significance ranking.

[0058] According to an embodiment of the present invention, the preset threshold of each disease branch network is set in the following manner: T1. Obtain a test set with the same distribution as the application scenario, and use the constructed multi-disease referral model to give the score of each sample in the current disease branch network in the test set; T2. Use the score of each sample as the threshold to be set to calculate the TPR and FPR corresponding to all scores, where TPR represents the true positive rate corresponding to the test set, and FPR represents the false positive rate corresponding to the test set; T3. Set the preset threshold of the current disease branch network to the threshold to be set that maximizes TPR + 1 - FPR.

[0059] The model of the present invention is trained under the constraint of two loss functions, which can be compatible with the concurrent multi-label situation. Moreover, the thresholds of each disease type in the system of the present invention can be adjusted independently (in contrast, in methods 2 and 3 in the background technology, the Softmax multi-classification cannot adjust the threshold, and generally, the category with the largest output is directly taken). The output of the samples of a specific label in other branches is supervised, and the situation where the output is unrestricted in method 2 will not occur. The output restriction on other branches is relative and not overly strict, so that the model will not overfit even if the sample annotation is incomplete.

[0060] The following will illustrate the effects of the present invention in combination with experimental data.

[0061] A simple experiment was conducted on fundus images. The data was divided into 5 categories: normal fundus (Normal), hypertensive retinopathy (HR), age-related macular degeneration (AMD), diabetic retinopathy (DR), and retinal vein occlusion (RVO). The annotations were all single-annotations without co-existing lesions, and the output was a single result. The effectiveness was illustrated by comparing the sensitivity, specificity, and accuracy of different methods in lesion recognition. Sensitivity, specificity, and accuracy are commonly used evaluation indicators. Among them, sensitivity: TP / (TP + FN), which indicates the proportion of all positive cases recalled as positive. The higher the sensitivity, the fewer missed diagnoses; specificity: TN / (TN + FP), which indicates the proportion of all negative cases judged as negative. The higher the specificity, the fewer misdiagnoses; accuracy: TP / (TP + FP), which indicates the proportion of all recalled positive cases that are truly positive, and it is affected by the sample distribution.

[0062] Method A: Training with Sigmoid Loss, without considering the confounding relationship between different lesions. The experimental results of this method are shown in Table 1 as follows:

[0063] Table 1

[0064]

[0065]

[0066] It can be seen from Table 1 that the referral index in Method A is relatively high, while the single index is low, which indicates that...

[0067] Method B: Training with Softmax Loss, considering different lesions as mutually exclusive categories. The experimental results of this method are shown in Table 2 as follows:

[0068] Table 2

[0069]

[0070] It can be seen from Table 2 that the referral index in Method B is low, but the single index is high.

[0071] Method C: The present invention. The experimental results are shown in Table 3 as follows:

[0072]

[0073] It can be seen from Table 3 that both the referral index and the single index in the present invention are high.

[0074] From the above experimental data, it can be seen that the model obtained by Method A has high discrimination ability in the referral task, but has poor discrimination ability when giving specific referral reasons, mainly manifested in easily classifying HR as DR and RVO as HR and DR. Although Method B has greatly improved the model's ability to distinguish different diseases, it sacrifices the performance of the main task, namely referral. The present invention comprehensively surpasses Method A in terms of indicators, that is, on the premise of not decreasing the referral performance, it improves the model's ability to distinguish different diseases. Although the discrimination is slightly lower than that of Method B, the method of the present invention is compatible with multi-labeling and multi-output, and can achieve higher performance when extended to concurrent situations.

[0075] It should be noted that although the above steps are described in a specific order, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even the order can be changed, as long as the required functions can be achieved.

[0076] The present invention can be a system, method, and / or computer program product. The computer program product can include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0077] The computer-readable storage medium can be a tangible device that retains and stores instructions for use by an instruction execution device. The computer-readable storage medium can, for example, include but is not limited to an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punched card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing.

[0078] The embodiments of the present invention have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to technologies in the market, or to enable other ordinary skilled persons in the technical field to understand the disclosed embodiments.

Claims

1. A method for training a multi-disease referral system, characterized in that, The method includes: S1. Obtain training samples of fundus images for training, and label the fundus images with all diseases corresponding to each sample as positive labels to obtain a training dataset; S2. Input the training samples in the training dataset into the multi-disease referral system to be trained. Among them, the multi-disease referral system includes multiple disease branch networks. Each disease branch network includes a classifier corresponding to a disease and an activation layer for processing the output of the classifier. Each disease branch network outputs its corresponding score and whether referral is needed according to the input fundus image; the multi-disease referral system also includes a normalization layer for calculating the outputs of the classifiers of all disease branch networks, which is used to calculate the outputs of the classifiers of all disease branch networks to obtain the probability of the target object suffering from each disease and sort the probabilities according to the disease severity, where the score output by the disease branch network is used as the basis for sorting the disease severity; S3. Update the weights of the multiple disease branch networks according to the loss function of the training samples until convergence; where the loss function includes the losses of all disease branch networks plus the multi-label regression loss of the merged outputs of the classifiers of all disease branch networks.

2. The method according to claim 1, wherein The multi-disease referral system also includes a backbone convolutional neural network, and its output is respectively connected to each disease branch network.

3. The method according to claim 1, wherein The activation layer of each disease branch network is a sigmoid layer, and the normalization layer is a Softmax layer.

4. The method according to any one of claims 1-3, characterized in that, The loss function is expressed as: L = ∑L1 + L2 where L represents the total loss of the multi-disease referral system, ∑L1 is the sum of the losses of each disease branch network, each L1 is the loss generated by samples containing positive labels of the disease class corresponding to the current disease branch network and healthy samples, and L2 is the multi-label regression loss of the merged outputs of the classifiers of all disease branch networks.

5. The method according to claim 4, wherein The loss of each disease branch network is the binary cross-entropy loss, which is expressed as: Among them, x represents the sample fundus image input into the disease branch network, and f i (x) represents the predicted output of x in the disease branch network i.

6. The method according to claim 4, wherein Among them, x represents the sample fundus image input into the disease branch network, and f i (x) represents the output of the disease branch network i, and P represents the set of all positive labels contained in the sample x.

7. The method according to claim 4, wherein Among them, x represents the sample fundus image input into the disease branch network, and f i (x) represents the predicted output of x in the disease branch network i, and P represents the set of all positive labels included in the sample x.

8. A multi-disease referral system trained by the method according to any one of claims 1-7, the system includes: Multiple disease branch networks, where each disease branch network includes a classifier corresponding to a disease and an activation layer for processing the output of the classifier. Each disease branch network outputs its corresponding score and whether referral is needed according to the input fundus image; A normalization layer for calculating the outputs of the classifiers of all disease branch networks to obtain the probability of the target object suffering from each disease and sorting the probabilities according to the disease severity, where the score output by the disease branch network is used as the basis for sorting the disease severity.

9. The system according to claim 8, wherein The activation layer of each disease branch network outputs the score of the input fundus image in the current disease branch network, and judges whether referral is needed based on a preset threshold; Among them, the result of whether referral is needed is obtained by comparing the score output by the current disease branch network with the preset threshold of this disease branch network. If the score of the input fundus image is greater than or equal to the preset threshold of this disease branch network, it is determined that referral is needed.

10. The system according to claim 9, wherein, The preset threshold of each disease branch network is set in the following manner: T1. Obtain a test set with the same distribution as the application scenario, and use the constructed multi-disease referral system to give the score of each sample in the test set in the current disease branch network; T2. Calculate the TPR and FPR corresponding to all rate scores with the score of each sample as the threshold to be set, where TPR represents the true positive rate corresponding to the test set, and FPR represents the false positive rate corresponding to the test set; T3. Set the preset threshold of the current disease branch network as the threshold to be set that maximizes TPR + 1 - FPR.

11. A multi-disease referral method, the method comprising: P1. Obtain the fundus image of the patient; P2. Use the system according to any one of claims 8-10 to determine whether the patient needs referral and the disease probabilities sorted by disease significance; P3. Refer the patient according to the disease probabilities sorted by disease significance.

12. A computer-readable storage medium, characterized in that, It includes a computer program, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 7.

13. An electronic device, characterized in that, Comprising: One or more processors; A storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device is caused to implement the steps of the method according to any one of claims 1 to 7.

Images