Medical image annotation method based on uncertainty enhancement and active learning optimization

Abstract

The invention provides a medical image annotation method based on uncertainty enhancement and active learning optimization. The method comprises the steps of obtaining a to-be-annotated medical image; inputting the to-be-labeled medical image into a pre-training labeling model for labeling processing to obtain a medical labeling result and a corresponding uncertainty labeling score; judging whether the uncertainty annotation score meets an annotation score threshold value or not; when the uncertainty annotation score is smaller than or equal to an annotation score threshold value, the medical annotation result serves as a final annotation result; wherein the pre-training annotation model is provided with a main task branch and a cognitive uncertainty prediction branch, the main task branch is used for carrying out image annotation processing on a medical image to be annotated, and the cognitive uncertainty prediction branch is used for carrying out uncertainty judgment processing on a medical annotation result output by the main task branch; and the loss function of the cognitive uncertainty prediction branch takes the loss value of the main task branch as input. The dependence on large-scale annotation data and the calculation time consumption are reduced, and the annotation efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image automatic analysis, and in particular to a medical image labeling method based on uncertainty enhancement and active learning optimization. BACKGROUND

[0002] In recent years, with the breakthrough of deep learning and machine learning technology, the importance of medical image automatic analysis in medical informatization is increasingly prominent. Medical image processing, classification and recognition technology (such as lesion detection, lesion classification, etc.) as the core application direction, highly depends on large-scale labeled data to support model training. However, the vast amount of unlabeled medical images accumulated by medical institutions need the participation of professional medical staff in labeling, which is time-consuming and costly, directly limiting the rapid construction and clinical deployment of high-performance analysis systems. Under this background, how to efficiently screen the subset with the greatest gain for model training from the vast amount of unlabeled data has become a key technical challenge to reduce labeling cost and improve training efficiency.

[0003] To solve the above problems, the existing technology has formed a solution based on active learning framework and uncertainty evaluation technology. Active learning selects the most informative unlabeled samples for labeling through iteration, gradually optimizing the model performance; uncertainty evaluation technology quantifies the confidence of model prediction to assist in identifying samples with higher value for model training (such as samples with low prediction confidence). Typical methods include uncertainty modeling based on Bayesian neural networks, using Monte Carlo Dropout (MC Dropout) technology to estimate the prediction distribution, etc. These technologies try to quantify the "cognitive uncertainty" of the model to the sample through statistical or probabilistic means, thereby guiding the sample selection strategy.

[0004] However, the existing technology still has significant limitations: first, the strong dependence of deep learning models on large-scale labeled data has not been fundamentally solved, and the initial training and iterative process still requires continuous labeling of new samples, which is difficult to quickly deploy in resource-limited scenarios; second, the computational efficiency of uncertainty evaluation cannot meet the clinical needs, and methods based on Bayesian neural networks or Monte Carlo Dropout require multiple forward propagations or complex probability calculations to estimate uncertainty, significantly increasing the analysis time of high-resolution medical images (such as whole-slide pathology images), limiting large-scale deployment; third, the sample selection strategy of active learning is not intelligent enough, traditional methods mostly use simple rules (such as random selection, confidence threshold screening), lacking the ability to accurately identify "high information content samples", and cannot fully exploit the characteristics of samples, resulting in low efficiency of active learning and difficulty in effectively reducing labeling requirements and accelerating model convergence. SUMMARY

[0005] In order to solve the above problems existing in the prior art, the present application provides a medical image labeling method based on uncertainty enhancement and active learning optimization.

[0006] The technical problem to be solved by the present application is realized by the following technical scheme: In a first aspect, the present application provides a medical image labeling method based on uncertainty enhancement and active learning optimization, comprising: obtaining a medical image to be labeled; inputting the medical image to be labeled into a pre-trained labeling model for labeling processing to obtain a medical labeling result and a corresponding uncertainty labeling score; determining whether the uncertainty labeling score meets a labeling score threshold; when the uncertainty labeling score is less than or equal to the labeling score threshold, taking the medical labeling result as a final labeling result; wherein the pre-trained labeling model is updated in parameters by an active learning method and a medical image sample with a low uncertainty labeling score, the pre-trained labeling model is provided with a main task branch and a cognitive uncertainty prediction branch, the main task branch is used for image labeling processing on the medical image to be labeled, the cognitive uncertainty prediction branch is used for uncertainty determination processing on the medical labeling result output by the main task branch, and a loss function of the cognitive uncertainty prediction branch takes a loss value of the main task branch as input.

[0007] Optionally, the pre-trained labeling model comprises: a pre-trained model feature representation module, a pre-trained recognition head module and a pre-trained uncertainty head module; the pre-trained model feature representation module and the pre-trained recognition head module are connected in series to constitute the main task branch; the pre-trained model feature representation module and the pre-trained uncertainty head module are connected in series to constitute the cognitive uncertainty prediction branch.

[0008] Optionally, the pre-trained uncertainty head module comprises: a first linear layer, a second linear layer, a third linear layer and a fourth linear layer; a feature dimension of the first linear layer is ; a feature dimension of the second linear layer and the third linear layer is ; a feature dimension of the fourth linear layer is ; the first linear layer, the second linear layer and the third linear layer all adopt a RELU activation function; the fourth linear layer adopts a Softplus activation function; the Softplus activation function is expressed as: ; wherein, denotes an uncertainty annotation score, denotes a medical image to be annotated corresponding uncertainty annotation score, denotes a Softplus activation function.

[0009] Optionally, after determining whether the uncertainty annotation score meets the annotation score threshold, the medical image annotation method based on uncertainty enhancement and active learning optimization further comprises: When the uncertainty annotation score is greater than the annotation score threshold, the medical image to be annotated is sent to an expert annotation port, and a manual annotation result of the expert annotation port is obtained; The manual annotation result is taken as a final annotation result.

[0010] Optionally, the loss function corresponding to the cognitive uncertainty prediction branch is represented as: ; wherein, denotes a value of the loss function corresponding to the cognitive uncertainty prediction branch, denotes a current medical image sample, denotes a current medical image sample corresponding loss value of the main task branch, denotes a current medical image sample corresponding uncertainty annotation score in the cognitive uncertainty prediction branch, denotes a current medical image sample total number of sample images corresponding to the batch in which the current medical image sample is located.

[0011] Optionally, the training process of the pre-training annotation model comprises: S201, obtaining a medical image sample dataset; the medical image sample dataset comprises: a current first sample dataset and a current second sample dataset; the current first sample dataset is a labeled medical image, and the current second sample dataset is an unlabeled medical image; S202, training the current main task branch based on the current first sample dataset, and taking the current main task branch meeting a first convergence condition as a current pre-training main task branch; S203, performing parameter freezing processing on the current pre-training main task branch, training the current cognitive uncertainty prediction branch by using the current first sample dataset, and taking the current cognitive uncertainty prediction branch meeting a second convergence condition as a current pre-training cognitive uncertainty prediction branch; S204, jointly constructing the current pre-training main task branch and the current pre-training cognitive uncertainty prediction branch into a current annotation model; S205, select a plurality of sample images from the current second sample data set as a current third sample data set, input the current third sample data set into the current labeling model, obtain the uncertainty labeling sample score corresponding to the current pre-training cognitive uncertainty prediction branch and the current pre-training main task branch, and the current third sample labeling data set; S206, sort the uncertainty labeling sample score in descending order to obtain a labeling sorting result, and send the current third sample data set corresponding to the first M labeling sorting results to the artificial labeling end for data labeling to obtain an updated labeling data set; S207, combine the updated labeling data set, the current third sample labeling data set and the current first sample data set as the current first sample data set in S202, and remove the current third sample data set from the current second sample data set as the current second sample data set in S202; S208, re-execute S202-S207 until the sample images in the current second sample data set are empty, and the current labeling model in S204 at this time is taken as a pre-training labeling model.

[0012] Optionally, the first convergence condition comprises: the value of the loss function of the current main task branch is continuously less than the first loss threshold, or the training number of the current main task branch is greater than the first preset training number threshold; The second convergence condition comprises: the value of the loss function of the current cognitive uncertainty prediction branch is continuously less than the second loss threshold, or the training number of the current cognitive uncertainty prediction branch is greater than the second preset training number threshold; The structure of the current main task branch is the same as that of the main task branch, and the structure of the current cognitive uncertainty prediction branch is the same as that of the cognitive uncertainty prediction branch.

[0013] In a second aspect, the present application provides a medical image labeling device based on uncertainty enhancement and active learning optimization, which comprises an acquisition unit, a labeling unit, a judgment unit and a determination unit. The acquisition unit is used for acquiring a medical image to be labeled. The labeling unit is used for inputting the medical image to be labeled into a pre-training labeling model for labeling processing to obtain a medical labeling result and a corresponding uncertainty labeling score. The judgment unit is used for judging whether the uncertainty labeling score meets a labeling score threshold. The determination unit is used for taking the medical labeling result as a final labeling result when the uncertainty labeling score is less than or equal to the labeling score threshold. The pre-training annotation model is updated in parameters by using an active learning method and a medical image sample with a low uncertainty annotation score, the pre-training annotation model is provided with a main task branch and a cognitive uncertainty prediction branch, the main task branch is used for image annotation processing on the medical image to be annotated, the cognitive uncertainty prediction branch is used for uncertainty judgment processing on the medical annotation result output by the main task branch, and a loss function of the cognitive uncertainty prediction branch takes a loss value of the main task branch as input.

[0014] Optionally, the pre-training annotation model comprises: a pre-training model feature representation module, a pre-training recognition head module and a pre-training uncertainty head module; the pre-training model feature representation module and the pre-training recognition head module are connected in series to form the main task branch; the pre-training model feature representation module and the pre-training uncertainty head module are connected in series to form the cognitive uncertainty prediction branch.

[0015] In a third aspect, the present application provides a medical image annotation device based on uncertainty enhancement and active learning optimization, comprising a processor, a storage medium and a bus, the storage medium stores machine readable instructions executable by the processor, when the medical image annotation device based on uncertainty enhancement and active learning optimization is running, the processor communicates with the storage medium through the bus, and the processor executes the machine readable instructions to perform the steps of the medical image annotation method based on uncertainty enhancement and active learning optimization according to any one of the above first aspect.

[0016] The application provides a medical image labeling method based on uncertainty enhancement and active learning optimization, comprising the following steps: obtaining a medical image to be labeled; inputting the medical image to be labeled into a pre-trained labeling model for labeling processing to obtain a medical labeling result and a corresponding uncertainty labeling score; determining whether the uncertainty labeling score meets a labeling score threshold; when the uncertainty labeling score is less than or equal to the labeling score threshold, taking the medical labeling result as a final labeling result; wherein the pre-trained labeling model is updated in parameters by using an active learning method and a medical image sample with a low uncertainty labeling score, the pre-trained labeling model is provided with a main task branch and a cognitive uncertainty prediction branch, the main task branch is used for image labeling processing on the medical image to be labeled, the cognitive uncertainty prediction branch is used for uncertainty determination processing on the medical labeling result output by the main task branch, and a loss function of the cognitive uncertainty prediction branch takes a loss value of the main task branch as input. In the application, firstly, a double-branch joint model is constructed: the main task branch performs medical image labeling, and the cognitive uncertainty prediction branch synchronously generates an uncertainty score through its loss function (taking the loss value of the main branch as input); then, during labeling processing, the model can synchronously output a medical labeling result and a corresponding uncertainty score; secondly, high-uncertainty samples are automatically screened by setting a labeling score threshold, only these samples need to be manually audited, and the manual labeling range is greatly reduced; at the same time, uncertainty evaluation can be completed by single forward propagation without repeated calculation of the traditional probability method. Finally, the dependence on large-scale labeling data and the calculation time are significantly reduced, the high-resolution medical image labeling efficiency is improved, and the rapid deployment of the resource-limited scene is supported.

[0017] The application will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A flowchart of a medical image labeling method based on uncertainty enhancement and active learning optimization provided by the embodiment of the application is shown; Figure 2 A schematic diagram of prediction results of a model trained on the basis of a labeled data set of each round of a medical image tested in a 5-round iteration process is exemplarily shown; Figure 3 A comparison result of the existing method and the method of the application in the recognition accuracy is exemplarily shown; Figure 4 A comparison result of the existing method and the method of the application in the operation efficiency is exemplarily shown; Figure 5 A structural schematic diagram of a medical image labeling device based on uncertainty enhancement and active learning optimization provided by the embodiment of the application is shown; Figure 6A structural schematic diagram of a medical image labeling device based on uncertainty enhancement and active learning optimization is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0019] The present application will be further described in conjunction with specific embodiments, but the embodiments of the present application are not limited thereto.

[0020] In order to reduce the dependence on large-scale labeled data and time-consuming computation, and improve the labeling efficiency of high-resolution medical images, an embodiment of the present application provides a medical image labeling method based on uncertainty enhancement and active learning optimization. Figure 1 A flowchart of a medical image labeling method based on uncertainty enhancement and active learning optimization is provided for an embodiment of the present application, as shown in Figure 1 , which includes: S101, obtaining a medical image to be labeled.

[0021] S102, inputting the medical image to be labeled into a pre-trained labeling model for labeling processing to obtain a medical labeling result and a corresponding uncertainty labeling score.

[0022] Optionally, in the present embodiment, the medical labeling result is a medical image containing lesion position information and lesion type. The uncertainty labeling score is an estimated score of whether the lesion position information and the lesion type contained in the above medical labeling result are accurate.

[0023] Optionally, the pre-trained labeling model includes: a pre-trained model feature representation module, a pre-trained recognition head module, and a pre-trained uncertainty head module; the pre-trained model feature representation module and the pre-trained recognition head module are connected in series to form a main task branch; the pre-trained model feature representation module and the pre-trained uncertainty head module are connected in series to form a cognitive uncertainty prediction branch.

[0024] In the present embodiment, the main task branch is responsible for performing the core medical image classification or detection task.

[0025] The cognitive uncertainty prediction branch is a lightweight regression network composed of multiple linear layers. The branch is a regression model that takes the loss of the main task branch as a proxy indicator. Through the regression model, the loss value that the main task branch will produce for the same input sample is predicted, thereby quantifying the cognitive uncertainty of the pre-trained labeling model.

[0026] Optionally, the pre-trained uncertainty head module includes: a first linear layer, a second linear layer, a third linear layer, and a fourth linear layer; The feature dimension of the first linear layer is dimension of the second linear layer and the third linear layer is dimension of the fourth linear layer is dimension of the fourth linear layer is The first linear layer, the second linear layer and the third linear layer all adopt a RELU activation function. The fourth linear layer adopts a Softplus activation function. The Softplus activation function is expressed as: ; wherein, denotes an uncertainty annotation score, denotes a medical image to be annotated corresponding uncertainty annotation score, denotes a Softplus activation function.

[0027] S103, determining whether the uncertainty annotation score meets an annotation score threshold.

[0028] Optionally, after S103, further comprising: When the uncertainty annotation score is greater than the annotation score threshold, the medical image to be annotated is sent to an expert annotation port, and a manual annotation result of the expert annotation port is obtained; The manual annotation result is taken as a final annotation result.

[0029] S104, when the uncertainty annotation score is less than or equal to the annotation score threshold, the medical annotation result is taken as a final annotation result.

[0030] wherein the pre-training annotation model adopts an active learning method and a medical image sample with a low uncertainty annotation score to update parameters, the pre-training annotation model is provided with a main task branch and a cognitive uncertainty prediction branch, the main task branch is used for image annotation processing on the medical image to be annotated, the cognitive uncertainty prediction branch is used for uncertainty determination processing on the medical annotation result output by the main task branch, and a loss function of the cognitive uncertainty prediction branch takes a loss value of the main task branch as input.

[0031] Optionally, the loss function corresponding to the cognitive uncertainty prediction branch is expressed as: ; wherein, denotes a value of the loss function corresponding to the cognitive uncertainty prediction branch, denotes a current medical image sample, denotes a loss value of the main task branch corresponding to the current medical image sample denotes a loss value of the main task branch corresponding to the current medical image sample denotes a loss value of the main task branch corresponding to the current medical image sample corresponding to the uncertainty prediction branch of the cognitive uncertainty prediction branch, representing the current medical image sample total number of sample images corresponding to the batch.

[0032] Optionally, the training process of the pre-training labeling model comprises: S201, acquiring a medical image sample data set; the medical image sample data set comprises: a current first sample data set and a current second sample data set; the current first sample data set is a labeled medical image, and the current second sample data set is an unlabeled medical image; S202, training the current main task branch based on the current first sample data set, and taking the current main task branch satisfying a first convergence condition as a current pre-training main task branch; S203, performing parameter freezing processing on the current pre-training main task branch, training the current cognitive uncertainty prediction branch by using the current first sample data set, and taking the current cognitive uncertainty prediction branch satisfying a second convergence condition as a current pre-training cognitive uncertainty prediction branch; S204, jointly constructing the current labeling model by using the current pre-training main task branch and the current pre-training cognitive uncertainty prediction branch; S205, selecting a plurality of sample images from the current second sample data set as a current third sample data set, inputting the current third sample data set into the current labeling model, and acquiring uncertainty labeling sample scores corresponding to the current pre-training cognitive uncertainty prediction branch and the current pre-training main task branch and the current third sample labeling data set; S206, sorting the uncertainty labeling sample scores in descending order to obtain a labeling sorting result, and sending the current third sample data set corresponding to the first M labeling sorting results to a manual labeling end for data labeling to obtain an updated labeling data set; S207, combining the updated labeling data set, the current third sample labeling data set and the current first sample data set as the current first sample data set in S202, and taking the current second sample data set without the current third sample data set as the current second sample data set in S202; S208, re-executing S202-S207 until the sample images in the current second sample data set are empty, and taking the current labeling model in S204 at this time as a pre-training labeling model.

[0033] It can be understood that, in the embodiment of the application, by sorting the uncertainty annotation sample scores in descending order, and sending the top M annotation sorting results corresponding to the current third sample data set with high uncertainty to the artificial annotation end for data annotation, the accuracy of the current third sample annotation data set annotation result can be improved, finally the updated annotation data set, the current third sample annotation data set and the current first sample data set are combined, the current main task branch and the current pre-training main task branch are trained, the overall current annotation model can be updated based on high-quality samples at any time, and the annotation and recognition accuracy of the final pre-training annotation model is improved.

[0034] That is, in order to ensure the independence and performance of the modules, the application adopts a two-stage sequential bypass training strategy as follows: The first stage (main task training): concentrate computing resources, and only fully train the main task branch and the shared main network (initial model feature representation module) until the performance of the main task branch converges on the validation set.

[0035] The second stage (uncertainty prediction task training): freeze all network parameters trained in the first stage, and only train the initial uncertainty head module. The unique feature of this stage is that the real classification loss generated by the main task branch for each sample is used as a supervision signal, and the parameters of the initial uncertainty head module are optimized by minimizing the loss function corresponding to the uncertainty prediction branch. This asynchronous optimization mechanism converts the evaluation of cognitive uncertainty into an efficient and specific regression task.

[0036] The overall training and application process of the pre-training annotation model will be described below: Suppose the starting unlabeled data set is D, the current annotated data set is S, and the amount of data selected for annotation each time is d.

[0037] 1. First iteration: randomly select d data from D. Manually annotate the d data. At this time, D is updated to D-d, and S is updated to S+d. After annotation, the initial annotation model is trained on the updated S. After the initial annotation model is trained, the cognitive uncertainty prediction branch is used to infer the remaining data in D, and the remaining data in D is sorted according to the output uncertainty annotation score.

[0038] 2. Subsequent iteration: After sorting the uncertainty annotation scores of the remaining data in D, select the d data with the highest uncertainty. Manually correct the model annotation results of the d data to form new annotations. Repeat the above steps: update D and S, and continue to train the current annotation model on the updated S, and infer and sort the remaining D.

[0039] 3. Termination condition: the iteration process continues until the performance of the current annotation model reaches the expected level, or the unlabeled data D is completely annotated.

[0040] To verify the effect of the medical image annotation method based on uncertainty enhancement and active learning optimization provided by the embodiments of the present application, Figure 2 The prediction result schematic diagram of the model trained on the annotated data set of each round of the test medical image in the 5-round iteration process is exemplarily shown. As Figure 2 shown, yellow is the uncertainty area, and blue is the negative area. After 5 rounds of iteration, the range of the yellow area gradually decreases and approaches zero, confirming that the model continues to improve the prediction ability of unseen samples with effective newly added annotated data training.

[0041] Optionally, the first convergence condition comprises: the value of the loss function of the current main task branch is continuously less than the first loss threshold value, or the training number of the current main task branch is greater than the first preset training number threshold value; The second convergence condition comprises: the value of the loss function of the current cognitive uncertainty prediction branch is continuously less than the second loss threshold value, or the training number of the current cognitive uncertainty prediction branch is greater than the second preset training number threshold value; The structure of the current main task branch is the same as that of the main task branch, and the structure of the current cognitive uncertainty prediction branch is the same as that of the cognitive uncertainty prediction branch.

[0042] To further illustrate the effectiveness of the method of the present application, simulation experiments were also conducted on comparative algorithms, as follows: Figure 3 The comparative results of the existing method and the method of the present application in recognition accuracy are exemplarily shown. The lowest confidence method, the random baseline method, and the MC-Dropout method were compared with the method of the present application, and the same "difficult sample" was annotated or recognized by using the above methods, wherein, Figure 3 The steeper the curve in the graph, the more accurately the method can find the samples that the model will predict incorrectly. The experimental results show that the performance curve of the method of the present application (green solid line) is always significantly better than other comparative methods, proving its excellent performance in identifying high-value samples.

[0043] Figure 4 The comparative results of the existing method and the method of the present application in operation efficiency are exemplarily shown. As Figure 4 shown, while achieving the best effect, the calculation time of the method of the present application is in the same order of magnitude as the simplest "lowest confidence" method, and is tens of times faster than the advanced MC-Dropout method, proving its high efficiency and feasibility in the application of large-scale clinical data sets.

[0044] The embodiment of the present application provides a medical image labeling method based on uncertainty enhancement and active learning optimization, comprising: obtaining a to-be-labeled medical image; inputting the to-be-labeled medical image into a pre-trained labeling model for labeling processing to obtain a medical labeling result and a corresponding uncertainty labeling score; determining whether the uncertainty labeling score meets a labeling score threshold; when the uncertainty labeling score is less than or equal to the labeling score threshold, regarding the medical labeling result as a final labeling result; wherein the pre-trained labeling model adopts an active learning method and a medical image sample with a low uncertainty labeling score to update parameters, the pre-trained labeling model is provided with a main task branch and a cognitive uncertainty prediction branch, the main task branch is used for image labeling processing on the to-be-labeled medical image, the cognitive uncertainty prediction branch is used for uncertainty judgment processing on the medical labeling result output by the main task branch, and a loss function of the cognitive uncertainty prediction branch takes a loss value of the main task branch as input. In the embodiment of the present application, firstly, a double-branch joint model is constructed: the main task branch performs medical image labeling, and the cognitive uncertainty prediction branch synchronously generates an uncertainty score through its loss function (taking the main branch loss value as input); then, during labeling processing, the model can synchronously output a medical labeling result and a corresponding uncertainty score; secondly, high-uncertainty samples are automatically screened by setting a labeling score threshold, only these samples need to be manually audited, and the manual labeling range is greatly reduced; at the same time, uncertainty evaluation can be completed through single forward propagation without repeated calculation of the traditional probability method. Finally, the dependence on large-scale labeling data and the calculation time are significantly reduced, the high-resolution medical image labeling efficiency is improved, and the rapid deployment of the resource-limited scene is supported.

[0045] In summary, the medical image labeling method based on uncertainty enhancement and active learning optimization provided by the embodiment of the present application has the following beneficial effects: 1. High calculation efficiency and clinical practicability: the decoupled architecture of the present application can obtain medical labeling results and cognitive uncertainty scores simultaneously through only one forward propagation, which fundamentally solves the bottleneck of high calculation complexity and inability to be applied to large-scale clinical practice caused by multiple reasoning of existing uncertainty evaluation methods (such as Monte Carlo Dropout).

[0046] 2. Significantly reduces labeling cost and labor cost: through the intelligent sample selection strategy based on cognitive uncertainty prediction, the present application can maximize the "information value" of each piece of labeling data, so as to achieve the same or even higher model accuracy with fewer labeling samples, thereby effectively relieving the labeling burden of medical experts.

[0047] 3. The model iteration and convergence efficiency is greatly improved: by focusing on learning the "high cognitive uncertainty" samples that can most improve the model in each iteration, the method can significantly accelerate the convergence speed of the model, shorten the development period, and save computing resources.

[0048] 4. The unique two-stage sequential training strategy ensures that the introduction of uncertainty prediction tasks does not interfere with the performance of the main task branch, ensuring the core value of the model in clinical applications.

[0049] 5. The reliability and human-computer collaboration of the clinical assistance task are enhanced: the uncertainty prediction branch can be seamlessly integrated into the actual clinical assistance workflow due to its high computational efficiency. During the reasoning phase, the method can highlight the areas or samples considered uncertain by the model in real time, prompting the doctor to review them in detail, and building a more reliable "AI+doctor" collaborative medical model.

[0050] The method provided by the embodiment of the application can be applied to an electronic device. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc., and the embodiment of the application is not limited thereto.

[0051] Based on the same inventive concept, the embodiment of the application also provides a medical image labeling device based on uncertainty enhancement and active learning optimization. Figure 5 As shown in the structure diagram of the medical image labeling device based on uncertainty enhancement and active learning optimization provided by the embodiment of the application, Figure 5 it includes an acquisition unit 501, a labeling unit 502, a judgment unit 503, and a determination unit 504. The acquisition unit 501 is configured to acquire a medical image to be labeled. The labeling unit 502 is configured to input the medical image to be labeled into a pre-trained labeling model for labeling processing to obtain a medical labeling result and a corresponding uncertainty labeling score. The judgment unit 503 is configured to determine whether the uncertainty labeling score meets a labeling score threshold. The determination unit 504 is configured to, when the uncertainty labeling score is less than or equal to the labeling score threshold, take the medical labeling result as a final labeling result. The pre-trained labeling model is updated in parameters by using an active learning method and a medical image sample with a low uncertainty labeling score, the pre-trained labeling model is provided with a main task branch and a cognitive uncertainty prediction branch, the main task branch is configured to perform image labeling processing on the medical image to be labeled, the cognitive uncertainty prediction branch is configured to perform uncertainty determination processing on the medical labeling result output by the main task branch, and the loss function of the cognitive uncertainty prediction branch takes the loss value of the main task branch as input.

[0052] Optionally, the pre-training annotation model comprises: a pre-training model feature representation module, a pre-training recognition head module, and a pre-training uncertainty head module; the pre-training model feature representation module and the pre-training recognition head module are connected in series to form a main task branch; the pre-training model feature representation module and the pre-training uncertainty head module are connected in series to form a cognitive uncertainty prediction branch.

[0053] Figure 6 A structural schematic diagram of a medical image annotation device based on uncertainty enhancement and active learning optimization provided by an embodiment of the present application, comprising: a processor 610, a storage medium 620, and a bus 630, the storage medium 620 stores machine readable instructions executable by the processor 610, when the medical image annotation device based on uncertainty enhancement and active learning optimization is running, the processor 610 and the storage medium 620 communicate through the bus 630, the processor 610 executes the machine readable instructions to execute the steps of the method embodiment. The specific implementation and technical effects are similar, and will not be repeated here.

[0054] The storage medium can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk memory. Optionally, the storage medium can also be at least one storage device located away from the aforementioned processor.

[0055] The aforementioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0056] It is to be understood that the terms "first", "second", and the like, used in the description and in the claims, are used as adjectives to distinguish between similar objects, and do not necessarily have a specific order or sequence. It is to be understood that the data so used in the description and in the claims can be interchanged, as appropriate, to refer to a similar one of the other elements. The embodiments described in the following examples do not represent all the implementations consistent with the present application. Instead, they are merely examples consistent with some aspects of the present application.

[0057] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific feature or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In the description of the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any appropriate manner in one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in the specification.

[0058] Although the present application is described herein in conjunction with various embodiments, those skilled in the art, with the benefit of the drawings and the disclosure, can understand and appreciate other variations of the disclosed embodiments that are within the scope of the claimed application. In the description of the specification, the word "comprise" does not exclude other components or steps, "a" or "one" does not exclude a plurality, and "plurality" means two or more, unless otherwise expressly specified. In addition, some measures are described in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0059] The above is a further detailed description of the present application in conjunction with specific preferred embodiments, and cannot be considered as limiting the specific implementation of the present application to these descriptions. For those skilled in the art, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, which should be considered as falling within the scope of protection of the present application.

Claims

1. A medical image annotation method based on uncertainty enhancement and active learning optimization, characterized in that, include: Acquire medical images to be labeled; The medical image to be labeled is input into a pre-trained labeling model for labeling processing to obtain medical labeling results and corresponding uncertainty labeling scores; Determine whether the uncertainty label score meets the label score threshold; When the uncertainty labeling score is less than or equal to the labeling score threshold, the medical labeling result is taken as the final labeling result; The pre-trained annotation model uses an active learning method and medical image samples with low uncertainty annotation scores for parameter updates. The pre-trained annotation model has a main task branch and a cognitive uncertainty prediction branch. The main task branch is used to perform image annotation processing on the medical image to be annotated, and the cognitive uncertainty prediction branch is used to perform uncertainty judgment processing on the medical annotation results output by the main task branch. The loss function of the cognitive uncertainty prediction branch takes the loss value of the main task branch as input.

2. The medical image annotation method based on uncertainty enhancement and active learning optimization according to claim 1, characterized in that, The pre-trained labeled model includes: Pre-trained model feature representation module, pre-trained recognition head module, and pre-trained uncertainty head module; The pre-trained model feature representation module and the pre-trained recognition head module are connected in series to form the main task branch; The pre-trained model feature representation module and the pre-trained uncertainty head module are connected in series to form the cognitive uncertainty prediction branch.

3. The medical image annotation method based on uncertainty enhancement and active learning optimization according to claim 2, characterized in that, The pre-trained uncertainty head module includes: The first linear layer, the second linear layer, the third linear layer, and the fourth linear layer; The feature dimension of the first linear layer is The feature dimensions of the second linear layer and the third linear layer are both 1. The feature dimension of the fourth linear layer is . dimension; The first linear layer, the second linear layer, and the third linear layer all use the ReLU activation function; The fourth linear layer uses the Softplus activation function; The Softplus activation function is expressed as follows: ； in, The uncertainty label score represents the score. Indicates medical images to be labeled The corresponding uncertainty label score, This indicates the Softplus activation function.

4. The medical image annotation method based on uncertainty enhancement and active learning optimization according to claim 1, characterized in that, After determining whether the uncertainty labeling score meets the labeling score threshold, the medical image labeling method based on uncertainty enhancement and active learning optimization further includes: When the uncertainty labeling score is greater than the labeling score threshold, the medical image to be labeled is sent to the expert labeling port, and the manual labeling result of the expert labeling port is obtained; The manual annotation results are used as the final annotation results.

5. The medical image annotation method based on uncertainty enhancement and active learning optimization according to claim 3, characterized in that, The loss function corresponding to the cognitive uncertainty prediction branch is expressed as follows: ； in, This represents the value of the loss function corresponding to the cognitive uncertainty prediction branch. This represents the current medical image sample. Indicates the current medical image sample The corresponding loss value of the main task branch, Indicates the current medical image sample Corresponding to the uncertainty label score in the cognitive uncertainty prediction branch, Indicates the current medical image sample The total number of sample images corresponding to the batch.

6. The medical image annotation method based on uncertainty enhancement and active learning optimization according to claim 1, characterized in that, The training process of the pre-trained labeled model includes: S201. Obtain a medical image sample dataset; the medical image sample dataset includes: a current first sample dataset and a current second sample dataset; the current first sample dataset consists of labeled medical images, and the current second sample dataset consists of unlabeled medical images; S202. Train the current main task branch based on the current first sample dataset, and take the current main task branch that satisfies the first convergence condition as the current pre-trained main task branch; S203. Freeze the parameters of the current pre-trained main task branch, train the current cognitive uncertainty prediction branch using the current first sample dataset, and take the current cognitive uncertainty prediction branch that satisfies the second convergence condition as the current pre-trained cognitive uncertainty prediction branch. S204. The current pre-trained main task branch and the current pre-trained cognitive uncertainty prediction branch are combined to form the current labeled model; S205. Select multiple sample images from the current second sample dataset as the current third sample dataset, input the current third sample dataset into the current annotation model, and obtain the uncertainty annotation sample scores and the current third sample annotation dataset corresponding to the current pre-trained cognitive uncertainty prediction branch and the current pre-trained main task branch. S206. Sort the uncertain labeled sample scores in descending order to obtain the labeling sorting results, and send the current third sample dataset corresponding to the first M labeled sorting results to the manual labeling end for data labeling to obtain the updated labeled dataset. S207. Merge the updated labeled dataset, the current third sample labeled dataset, and the current first sample dataset into the current first sample dataset in S202, and remove the current third sample dataset from the current second sample dataset into the current second sample dataset in S202. S208. Re-execute S202-S207 until the sample images in the current second sample dataset are empty, and use the current annotation model in S204 at this time as the pre-trained annotation model.

7. The medical image annotation method based on uncertainty enhancement and active learning optimization according to claim 6, characterized in that, The first convergence condition includes: the value of the loss function of the current main task branch is consistently less than a first loss threshold or the number of training iterations of the current main task branch is greater than a first preset training iteration threshold; The second convergence condition includes: the value of the loss function of the current cognitive uncertainty prediction branch is consistently less than the second loss threshold or the number of training iterations of the current cognitive uncertainty prediction branch is greater than the second preset training iterations threshold; The structure of the current main task branch is the same as the structure of the main task branch; the structure of the current cognitive uncertainty prediction branch is the same as the structure of the cognitive uncertainty prediction branch.

8. A medical image annotation device based on uncertainty enhancement and active learning optimization, characterized in that, The medical image annotation device based on uncertainty enhancement and active learning optimization includes: an acquisition unit, an annotation unit, a judgment unit, and a determination unit; The acquisition unit is used to acquire the medical image to be labeled; The annotation unit is used to input the medical image to be annotated into a pre-trained annotation model for annotation processing, and obtain medical annotation results and corresponding uncertainty annotation scores; The judgment unit is used to determine whether the uncertainty labeling score meets the labeling score threshold; The determining unit is used to take the medical annotation result as the final annotation result when the uncertainty annotation score is less than or equal to the annotation score threshold; The pre-trained annotation model uses an active learning method and medical image samples with low uncertainty annotation scores for parameter updates. The pre-trained annotation model has a main task branch and a cognitive uncertainty prediction branch. The main task branch is used to perform image annotation processing on the medical image to be annotated, and the cognitive uncertainty prediction branch is used to perform uncertainty judgment processing on the medical annotation results output by the main task branch. The loss function of the cognitive uncertainty prediction branch takes the loss value of the main task branch as input.

9. The medical image annotation device based on uncertainty enhancement and active learning optimization according to claim 8, characterized in that, The pre-trained labeled model includes: Pre-trained model feature representation module, pre-trained recognition head module, and pre-trained uncertainty head module; The pre-trained model feature representation module and the pre-trained recognition head module are connected in series to form the main task branch; The pre-trained model feature representation module and the pre-trained uncertainty head module are connected in series to form the cognitive uncertainty prediction branch.

10. A medical image annotation device based on uncertainty enhancement and active learning optimization, characterized in that, include: The device includes a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the medical image annotation device based on uncertainty enhancement and active learning optimization is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the medical image annotation method based on uncertainty enhancement and active learning optimization as described in any one of claims 1-7.

Images