Image annotation method and device, equipment and storage medium
By automatically annotating medical images using an image annotation model and adjusting the model parameters after a credibility assessment, the problem of inconsistent annotation quality by doctors has been solved, achieving efficient and accurate image annotation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2026-01-14
- Publication Date
- 2026-05-01
AI Technical Summary
In existing technologies, the annotation of medical images relies on experienced doctors, which leads to inconsistent doctor qualifications and subjective differences, resulting in low annotation quality and efficiency, as well as high costs.
The image annotation model is used to divide and annotate medical images, and the credibility of the annotation results is evaluated. If the credibility is high, the result is directly output. If the credibility is low, the model parameters are adjusted and trained until the preset performance gain is achieved.
It improves the efficiency and accuracy of medical image annotation, reduces labor costs, and ensures the reliability of annotation results.
Smart Images
Figure CN121963205A_ABST
Abstract
Description
Image annotation methods, devices, equipment and storage media Technical Field
[0001] This application relates to the field of image recognition technology, and in particular to an image annotation method, apparatus, device and storage medium. Background Technology
[0002] In the era of rapid development of smart healthcare, medical imaging is becoming increasingly prevalent in medicine (e.g., X-ray images, computed tomography, magnetic resonance imaging, and ultrasound images). More and more departments are using imaging images as diagnostic aids, resulting in a growing number of medical images to be annotated. Currently, the annotation of medical images is mainly done by senior doctors. However, relying on human annotation has issues with varying doctor qualifications and subjective differences, leading to inconsistent quality of image annotation. In addition, the large number of medical images requires a large amount of manpower, resulting in high costs for medical image annotation.
[0003] Therefore, improving the efficiency and accuracy of medical image annotation is an urgent problem to be solved. Summary of the Invention
[0004] The main objective of this application is to provide an image annotation method, apparatus, device, and storage medium, which aims to improve the efficiency and accuracy of medical image annotation.
[0005] In a first aspect, this application provides an image annotation method, which includes the following steps: dividing and annotating each medical image in a medical image set using the image annotation model to obtain a medical annotation result for each medical image; evaluating the feasibility of image annotation for multiple medical image annotation results to obtain a comprehensive credibility; if the comprehensive credibility is greater than or equal to a preset credibility, then using the medical annotation result for each medical image as the target medical annotation result for each medical image; if the comprehensive credibility is less than the preset credibility, then adjusting the model parameters of the image annotation model and training the model with adjusted parameters according to the medical image set until a target image annotation model with a preset performance gain is obtained; dividing and annotating each medical image in the medical image set using the target image annotation model to obtain a target medical annotation result for each medical image.
[0006] Secondly, this application also provides an image annotation device, which includes an acquisition module, a generation module, a feasibility evaluation module, and a training module, wherein: the acquisition module is used to acquire a set of medical images to be annotated and an image annotation model, the set of images including at least one medical image; the generation module is used to perform image region division and annotation on each medical image in the set of medical images using the image annotation model, to obtain a medical annotation result for each medical image; the feasibility evaluation module is used to evaluate the feasibility of image annotation on the medical annotation results of multiple medical images, to obtain a comprehensive credibility; the generation module... The generation module is further configured to, if the overall confidence level is greater than or equal to a preset confidence level, use the medical annotation result of each medical image as the target medical annotation result for each medical image; the training module is configured to, if the overall confidence level is less than the preset confidence level, adjust the model parameters of the image annotation model, and train the model with adjusted parameters according to the medical image set until a target image annotation model with a preset performance gain is obtained; the generation module is further configured to, according to the target image annotation model, divide and annotate each medical image in the medical image set to obtain the target medical annotation result for each medical image.
[0007] Thirdly, this application also provides a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the steps of the image annotation method described above.
[0008] Fourthly, this application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the image annotation method described above.
[0009] This application provides an image annotation method, apparatus, device, and storage medium. The method involves acquiring a set of medical images to be annotated and an image annotation model, wherein the image set includes at least one medical image. The image annotation model is used to divide and annotate the image regions of each medical image in the set, resulting in a medical annotation result for each image. The feasibility of the image annotation is evaluated for the medical annotation results of multiple images to obtain a comprehensive credibility score. If the comprehensive credibility score is greater than or equal to a preset credibility score, the medical annotation result for each image is used as the target medical annotation result for that image. If the comprehensive credibility score is less than the preset credibility score, the model parameters of the image annotation model are adjusted, and the model with adjusted parameters is trained based on the medical image set until a target image annotation model with a preset performance gain is obtained. Finally, the target image annotation model is used to divide and annotate the image regions of each medical image in the set, resulting in a target medical annotation result for each image. This application uses an image annotation model to annotate a set of medical images to be annotated, which can accurately obtain medical annotation results. If the overall credibility of the medical annotation result is greater than or equal to the preset credibility, the medical annotation result is directly used as the target medical annotation result for each medical image and output, which greatly improves the efficiency of image annotation. If the overall credibility is less than the preset credibility, the model parameters of the image annotation model are adjusted, and the model with adjusted parameters is trained according to the medical image set until a target image annotation model with preset performance gain is obtained. Then, the target image annotation model is used to divide and annotate the medical image, which can accurately obtain the target medical annotation result for each medical image, greatly improving the accuracy of image annotation. Attached Figure Description
[0010] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1 is a flowchart illustrating an image annotation method according to an embodiment of this application; Figure 2 is a flowchart illustrating a sub-step of the image annotation method in Figure 1; Figure 3 is a flowchart illustrating another sub-step of the image annotation method in Figure 1; Figure 4 is a schematic block diagram illustrating an image annotation device according to an embodiment of this application; Figure 5 is a schematic block diagram illustrating the structure of a computer device according to an embodiment of this application.
[0012] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0014] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.
[0015] In the era of rapid development of smart healthcare, medical imaging is becoming increasingly prevalent in medicine (e.g., X-ray images, computed tomography, magnetic resonance imaging, and ultrasound images). More and more departments are using imaging images as diagnostic aids, resulting in a growing number of medical images to be annotated. Currently, the annotation of medical images is mainly done by senior doctors. However, relying on human annotation has issues with varying doctor qualifications and subjective differences, leading to inconsistent quality of image annotation. In addition, the large number of medical images requires a large amount of manpower, resulting in high costs for medical image annotation.
[0016] To address the aforementioned issues, this application provides an image annotation method, apparatus, device, and storage medium. The image annotation method includes acquiring a set of medical images to be annotated and an image annotation model, wherein the image set includes at least one medical image; performing image region division and annotation on each medical image in the set using the image annotation model to obtain a medical annotation result for each medical image; evaluating the feasibility of image annotation on the medical annotation results of multiple medical images to obtain a comprehensive credibility; if the comprehensive credibility is greater than or equal to a preset credibility, then using the medical annotation result of each medical image as the target medical annotation result for each medical image; if the comprehensive credibility is less than the preset credibility, then adjusting the model parameters of the image annotation model and training the parameter-adjusted model based on the medical image set until a target image annotation model with a preset performance gain is obtained; and performing image region division and annotation on each medical image in the set using the target image annotation model to obtain a target medical annotation result for each medical image.
[0017] The image annotation method can be applied to computer devices, such as mobile phones, tablets, laptops, desktop computers, personal digital assistants, and wearable devices.
[0018] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0019] Please refer to Figure 1, which is a flowchart illustrating an image annotation method provided by an embodiment of this application.
[0020] As shown in Figure 1, the image annotation method includes steps S101 to S106.
[0021] Step S101: Obtain the medical image set to be labeled and the image labeling model, wherein the image set includes at least one medical image.
[0022] The image set includes at least one medical image, which includes, but is not limited to, X-ray images, computed tomography images, magnetic resonance imaging images, and ultrasound images.
[0023] It should be noted that the image annotation model is a pre-trained image annotation model, which can be a large-scale multimodal model. The image annotation model is pre-trained based on a sample dataset, and the training process of the image annotation model can be described as the image annotation model adjustment process in the following embodiments. Moreover, the training process of the image annotation model is a conventional technical means, which will not be elaborated on in this application.
[0024] In some embodiments, a set of medical images to be labeled and an image labeling model are obtained. Obtaining the image labeling model facilitates more convenient labeling of each medical image in the set.
[0025] Step S102: Divide and annotate each medical image in the medical image set using the image annotation model to obtain the medical annotation result for each medical image.
[0026] The image annotation model includes an encoder layer, a modality fusion layer, and an output layer. The encoder layer is used to encode features of medical images, the modality fusion layer is used to perform cross-modal interactive fusion of image features, and the output layer is used to infer and predict the fused features of medical images to obtain medical annotation results.
[0027] It should be noted that this medical annotation result refers to the division and labeling of image regions in medical images. For example, if the medical image is a lung CT image, the lung CT image is divided into normal image regions and shadowed nodular image regions, and the size of the shadowed nodular image regions is labeled as 8 mm in diameter and referenced from historical case images.
[0028] In some embodiments, each medical image is processed by an encoder layer to extract and encode features, resulting in image features for each image. A modal fusion layer performs cross-modal interaction on these features, yielding fused features for each image. Finally, an output layer performs image region segmentation and annotation on the fused features, resulting in medical annotation results for each image. By sequentially segmenting and annotating image regions using the encoder, modal fusion, and output layers of this image annotation model, accurate medical annotation results for each medical image can be obtained.
[0029] Step S103: Evaluate the feasibility of medical annotation for multiple medical images to obtain a comprehensive credibility.
[0030] The overall credibility is determined based on the medical annotation results generated from each medical image in the set of medical images to be annotated.
[0031] In some embodiments, as shown in FIG2, step S103 includes sub-steps S1031 to S1032.
[0032] Sub-step S1031: Calculate the credibility of the medical annotation results of the multiple medical images to obtain the credibility value of each medical image.
[0033] In some embodiments, medical annotation results of multiple medical images are selected, and the credibility of the medical annotation results of the selected medical images is calculated to obtain the credibility value of each medical image.
[0034] In some embodiments, model confidence assessment, interpretability assessment, and cross-modal consistency assessment are performed on the medical annotation results of medical images to obtain model confidence values, interpretability assessment values, and cross-modal consistency assessment values. The model confidence values, interpretability assessment values, and cross-modal consistency assessment values are then weighted and summed to obtain the credibility value of the medical image. By performing model confidence assessment, interpretability assessment, and cross-modal consistency assessment on the medical annotation results, the credibility value of the medical image can be accurately obtained.
[0035] In some embodiments, the model confidence value can be obtained by evaluating the medical annotation results of medical images using the following method: obtaining the user's medical annotations of the medical images, obtaining the user's medical annotation results, calculating the similarity between the user's medical annotation results and the medical annotation results, and obtaining the similarity value as the model confidence value.
[0036] In some embodiments, the interpretability consistency assessment of medical annotation results for medical images can be obtained by determining the evaluation score of a preset recognition requirement standard that the medical annotation results of the medical image meet, and using this evaluation score as the interpretability consistency assessment value. The preset recognition requirement standard can be set according to actual conditions, and this application embodiment does not specifically limit it. For example, the preset recognition requirement standard can be set as image region segmentation requirements and recognition accuracy, etc.
[0037] In some embodiments, the evaluation score for determining the preset recognition requirement standard met by the medical annotation result of a medical image can be obtained by: acquiring a mapping table between the recognition requirement standard and the evaluation score, and querying the evaluation score corresponding to the recognition requirement standard from the mapping table. This mapping table is pre-established based on the recognition requirement standard and the evaluation score, and can be established according to actual circumstances; this embodiment does not specifically limit its implementation. The evaluation score can be accurately obtained through this mapping table.
[0038] In some embodiments, the cross-modal consistency assessment of medical annotation results for medical images can be performed to obtain a cross-modal consistency assessment value by: obtaining user medical annotations for medical images to obtain user medical annotation results, which include medical annotation results for each image region; comparing the user medical annotation results with the medical annotation results to determine the proportion of consistent and inconsistent regions, and using this proportion as the cross-modal consistency assessment value.
[0039] In some embodiments, the model confidence value, interpretability consistency assessment value, and cross-modal consistency assessment value of each medical image are normalized to obtain the target model confidence value, target interpretability consistency assessment value, and target cross-modal consistency assessment value for each medical image; a first weight parameter, a second weight parameter, and a third weight parameter are obtained; the target model confidence value and the first weight parameter are multiplied to obtain the first parameter; the target interpretability consistency assessment value and the second weight parameter are multiplied to obtain the second parameter; the target cross-modal consistency assessment value and the third weight parameter are multiplied to obtain the third parameter; the first parameter, the second parameter, and the third parameter are added together, and the resulting value is used as the confidence value of the medical image.
[0040] Sub-step S1032: Average the confidence values of multiple medical images to obtain the comprehensive confidence level.
[0041] The confidence values of multiple medical images are added together to obtain the total confidence value; the total confidence value is then divided by the number of medical images to obtain the overall confidence value.
[0042] Step S104: If the overall confidence level is greater than or equal to the preset confidence level, then the medical annotation result of each medical image is taken as the target medical annotation result of each medical image.
[0043] The preset confidence level can be set according to the actual situation, and this application embodiment does not impose specific limitations on it.
[0044] In some embodiments, if the overall confidence level is greater than or equal to a preset confidence level, the medical annotation result of each medical image is used as the target medical annotation result for each medical image. When the overall confidence level is greater than or equal to the preset confidence level, it indicates that the image annotation model has high recognition accuracy. Directly using the medical annotation result output by the image annotation model as the target medical annotation result for each medical image greatly improves the efficiency of image annotation.
[0045] Step S105: If the overall confidence level is less than the preset confidence level, the model parameters of the image annotation model are adjusted, and the model with adjusted parameters is trained according to the medical image set until the target image annotation model with preset performance gain is obtained.
[0046] If the overall credibility is less than the preset credibility, it indicates that the recognition accuracy of the image annotation model is low. In this case, it is necessary to adjust the model parameters of the image annotation model and retrain the image annotation model after adjusting the model parameters to obtain a more accurate image annotation model.
[0047] In some embodiments, as shown in FIG3, step S105 includes sub-steps S1051 to S1053.
[0048] Sub-step S1051: Adjust the model parameters of the image annotation model and select multiple medical images from the medical image set as training medical image images.
[0049] In some embodiments, the model parameters of the image annotation model are adjusted, and multiple medical image images are randomly selected from the medical image set as training medical image images.
[0050] In some embodiments, the model parameters of the image annotation model are adjusted, and medical image images with a confidence value lower than the overall confidence value are selected from the medical image set as training medical image images. The confidence value is determined by the medical annotation result obtained by the image annotation model from the image region division and annotation of the medical image images.
[0051] Sub-step S1052: Obtain the target medical annotation result corresponding to each of the training medical images, and construct a sample training data according to each of the training medical images and the corresponding target medical annotation result to obtain a sample training set, wherein the sample training set includes multiple sample training data.
[0052] Each training medical image is sent to the user, allowing the user to perform image region segmentation and annotation on each training medical image, resulting in target medical annotation results for each training medical image. The target medical annotation results sent by the user are then retrieved. Based on each training medical image and its corresponding target medical annotation results, a sample training data set is constructed to obtain the sample training set. By retrieving the target medical annotation results from the user's image region segmentation and annotation of the training medical images, and accurately constructing the sample training set for each training medical image and its corresponding target medical annotation results, a truly efficient method can be employed.
[0053] In some embodiments, training medical images are sent to the user so that the user can perform image region segmentation and annotation on the training medical images to obtain the target medical annotation result corresponding to each training medical image; the target medical annotation result corresponding to each training medical image sent by the user is then obtained. By sending training medical images to the user, the target medical annotation result of image region segmentation and annotation can be obtained in a timely manner.
[0054] Sub-step S1053: Adjust the model parameters of the image annotation model, and train the model with adjusted parameters according to the sample training set until a target image annotation model with preset performance gain is obtained.
[0055] The model parameters of the image annotation model are adjusted. A sample data is selected from the training set as the target sample data, which includes the training medical image and the target medical annotation result. The training medical image of the target sample data is input into the parameter-adjusted image annotation model to obtain the predicted image annotation result. Based on the target medical annotation result and the predicted image annotation result, it is determined whether the image annotation model can achieve the preset performance gain. If the image annotation model cannot achieve the preset performance gain, the model parameters of the image annotation model are adjusted again. A sample data is selected from the training set as the target sample data, which includes the training medical image and the target medical annotation result. The training medical image of the target sample data is input into the parameter-adjusted image annotation model to obtain the predicted image annotation result. Based on the target medical annotation result and the predicted image annotation result, it is determined whether the image annotation model can achieve the preset performance gain. If the image annotation model cannot achieve the preset performance gain, the process continues until a target image annotation model with the preset performance gain is obtained.
[0056] In some embodiments, determining whether the image annotation model can achieve a preset performance gain based on the target medical annotation result and the predicted image annotation result can be achieved by: calculating the similarity between the target medical annotation result and the predicted image annotation result to obtain a current similarity value; subtracting the current similarity value from the unit value to obtain the current loss value; obtaining historical loss values; calculating the average of the current loss value and the historical loss value to obtain a target loss value; if the target loss value is less than or equal to the preset loss value, determining that the image annotation model can achieve the preset performance gain; if the target loss value is greater than the preset loss value, determining that the image annotation model has failed to achieve the preset performance gain. The preset loss value can be set according to actual conditions, and this embodiment does not specifically limit it. For example, the preset loss value can be set to 0.02.
[0057] When the accuracy of image region segmentation and annotation using the image annotation model is insufficient, adjusting and training the parameters of the image annotation model can yield a more accurate image annotation model, thereby improving the accuracy of image annotation.
[0058] Step S106: Divide and annotate each medical image in the medical image set according to the target image annotation model to obtain the target medical annotation result for each medical image.
[0059] In some embodiments, after obtaining a more accurate target image annotation model, the target image annotation model is used to perform image region division and annotation on each medical image in the medical image set, resulting in the target medical annotation result for each medical image. Performing image region division and annotation on each medical image in the medical image set based on the target image annotation model makes the output target medical annotation result more accurate.
[0060] For example, the medical image set to be labeled includes 100,000 lung CT images. The 100,000 lung CT images are divided and labeled using an image labeling model to obtain a medical labeling result for each lung CT image. The feasibility of image labeling is evaluated for the medical labeling results of multiple lung CT images to obtain a comprehensive credibility. If the comprehensive credibility is greater than or equal to a preset credibility, the medical labeling result of each lung CT image is used as the target medical labeling result for each lung CT image. If the comprehensive credibility is less than the preset credibility, the model parameters of the image labeling model are adjusted, and the model with adjusted parameters is trained based on the 100,000 lung CT images until a target image labeling model with a preset performance gain is obtained. Based on the target image labeling model, each lung CT image is divided and labeled to obtain the target medical labeling result for each lung CT image.
[0061] The image annotation method provided in the above embodiments involves acquiring a set of medical images to be annotated and an image annotation model, wherein the set of images includes at least one medical image; dividing and annotating each medical image in the set of images using the image annotation model to obtain a medical annotation result for each medical image; evaluating the feasibility of image annotation for multiple medical image annotation results to obtain a comprehensive credibility; if the comprehensive credibility is greater than or equal to a preset credibility, then the medical annotation result for each medical image is used as the target medical annotation result for each medical image; if the comprehensive credibility is less than the preset credibility, then the model parameters of the image annotation model are adjusted, and the model with adjusted parameters is trained based on the set of medical images until a target image annotation model with a preset performance gain is obtained; and dividing and annotating each medical image in the set of images using the target image annotation model to obtain the target medical annotation result for each medical image. This application uses an image annotation model to annotate a set of medical images to be annotated, which can accurately obtain medical annotation results. If the overall credibility of the medical annotation result is greater than or equal to the preset credibility, the medical annotation result is directly used as the target medical annotation result for each medical image and output, which greatly improves the efficiency of image annotation. If the overall credibility is less than the preset credibility, the model parameters of the image annotation model are adjusted, and the model with adjusted parameters is trained according to the medical image set until a target image annotation model with preset performance gain is obtained. Then, the target image annotation model is used to divide and annotate the medical image, which can accurately obtain the target medical annotation result for each medical image, greatly improving the accuracy of image annotation.
[0062] Please refer to Figure 4, which is a schematic block diagram of an image annotation device provided in an embodiment of this application.
[0063] As shown in Figure 4, the image annotation device 200 includes an acquisition module 210, a generation module 220, a feasibility evaluation module 230, and a training module 240, wherein: the acquisition module 210 is used to acquire a set of medical images to be annotated and an image annotation model, wherein the image set includes at least one medical image; the generation module 220 is used to perform image region division and annotation on each medical image in the medical image set using the image annotation model, to obtain the medical annotation result for each medical image; the feasibility evaluation module 230 is used to evaluate the feasibility of image annotation on the medical annotation results of multiple medical images, to obtain a comprehensive credibility; the generation module 220 is used to perform image annotation feasibility evaluation on the medical annotation results of multiple medical images, to obtain a comprehensive credibility; the generation module 220 is used to perform image annotation feasibility evaluation on the medical image annotation results of multiple medical images, to obtain a comprehensive credibility; the generation module 220 is used to perform image region division and annotation on each medical image in the medical image set using the image annotation model, to obtain a feasibility evaluation module 230 ... The generation module 220 is further configured to, if the overall confidence level is greater than or equal to a preset confidence level, use the medical annotation result of each medical image as the target medical annotation result of each medical image; the training module 240 is configured to, if the overall confidence level is less than the preset confidence level, adjust the model parameters of the image annotation model, and train the model with adjusted parameters according to the medical image set until a target image annotation model with a preset performance gain is obtained; the generation module 220 is further configured to, according to the target image annotation model, divide and annotate each medical image in the medical image set to obtain the target medical annotation result of each medical image.
[0064] In some embodiments, the generation module 220 is further configured to: perform feature extraction and encoding processing on each of the medical image images through the encoder layer to obtain image features of each of the medical image images; perform cross-modal interaction on the image features of each of the medical image images through the modal fusion layer to obtain fused features of each of the medical image images; and perform image region division and annotation on the fused features of each of the medical image images through the output layer to obtain medical annotation results of each of the medical image images.
[0065] In some embodiments, the generation module 220 is further configured to: calculate the credibility of the medical annotation results of the plurality of medical image images to obtain the credibility value of each medical image image; and perform mean processing on the credibility values of all medical image images to obtain the comprehensive credibility.
[0066] In some embodiments, the generation module 220 is further configured to: perform model confidence assessment, interpretability consistency assessment, and cross-modal consistency assessment on the medical annotation results of the medical image to obtain model confidence value, interpretability consistency assessment value, and cross-modal consistency assessment value; and perform a weighted summation of the model confidence value, the interpretability consistency assessment value, and the cross-modal consistency assessment value to obtain the confidence value of the medical image.
[0067] In some embodiments, the training module 240 is further configured to: adjust the model parameters of the image annotation model, and select multiple medical images from the medical image set as training medical images; obtain the target medical annotation result corresponding to each training medical image, and construct a sample training data based on each training medical image and the corresponding target medical annotation result to obtain a sample training set, the sample training set including multiple sample training data; adjust the model parameters of the image annotation model, and train the model with adjusted parameters based on the sample training set until a target image annotation model with a preset performance gain is obtained.
[0068] In some embodiments, the training module 240 is further configured to: select medical image images with a confidence value less than the overall confidence value from the medical image set as training medical image images, wherein the confidence value is determined by the medical annotation result obtained by the image annotation model for image region division and annotation of the medical image images.
[0069] In some embodiments, the training module 240 is further configured to: send each of the training medical image images to the user, so that the user can perform image region division and annotation on the training medical image images to obtain the target medical annotation result corresponding to each of the training medical image images; and obtain the target medical annotation result corresponding to each of the training medical image images sent by the user.
[0070] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the above-mentioned image annotation device can be referred to the corresponding process in the aforementioned image annotation method embodiments, and will not be repeated here.
[0071] Please refer to Figure 5, which is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.
[0072] As shown in Figure 5, the computer device 300 includes a processor 302 and a memory 303 connected via a system bus 301. The memory 303 may include a storage medium and internal memory.
[0073] The storage medium may store a computer program. The computer program includes program instructions that, when executed, cause the processor to perform any image annotation method.
[0074] The processor 302 provides computing and control capabilities to support the operation of the entire computer device.
[0075] Internal memory provides an environment for the execution of computer programs stored in the storage medium. When the computer program is executed by the processor, it enables the processor to perform any image annotation method.
[0076] Those skilled in the art will understand that the structure shown in Figure 5 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or may combine certain components, or may have different component arrangements.
[0077] It should be understood that processor 302 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, the general-purpose processor can be a microprocessor or any conventional processor.
[0078] In one embodiment, the processor 302 is configured to run a computer program stored in a memory to perform the following steps: acquiring a set of medical images to be labeled and an image labeling model, wherein the set of images includes at least one medical image; performing image region division and labeling on each medical image in the set of images using the image labeling model to obtain a medical labeling result for each medical image; evaluating the feasibility of image labeling on the medical labeling results of multiple medical images to obtain a comprehensive credibility; if the comprehensive credibility is greater than or equal to a preset credibility, then using the medical labeling result of each medical image as the target medical labeling result for each medical image; if the comprehensive credibility is less than the preset credibility, then adjusting the model parameters of the image labeling model and training the model with adjusted parameters based on the set of medical images until a target image labeling model with a preset performance gain is obtained; performing image region division and labeling on each medical image in the set of images using the target image labeling model to obtain a target medical labeling result for each medical image.
[0079] In one embodiment, the image annotation model includes an encoder layer, a modality fusion layer, and an output layer. When the processor 302 performs image region segmentation and annotation on each medical image in the medical image set using the image annotation model to obtain the medical annotation result for each medical image, it performs the following: feature extraction and encoding processing on each medical image through the encoder layer to obtain image features for each medical image; cross-modal interaction of the image features of each medical image through the modality fusion layer to obtain fused features for each medical image; and image region segmentation and annotation of the fused features of each medical image through the output layer to obtain the medical annotation result for each medical image.
[0080] In one embodiment, when the processor 302 performs the image annotation feasibility evaluation on the medical annotation results of multiple medical image images to obtain a comprehensive credibility, it is configured to: calculate the credibility of the medical annotation results of multiple medical image images to obtain a credibility value for each medical image image; and perform mean processing on the credibility values of all medical image images to obtain the comprehensive credibility.
[0081] In one embodiment, when the processor 302 performs the confidence calculation of the medical annotation results of multiple medical image images to obtain the confidence value of each medical image image, it is configured to: perform model confidence assessment, interpretability consistency assessment, and cross-modal consistency assessment on the medical annotation results of the medical image images to obtain model confidence value, interpretability consistency assessment value, and cross-modal consistency assessment value; and perform a weighted summation of the model confidence value, the interpretability consistency assessment value, and the cross-modal consistency assessment value to obtain the confidence value of the medical image image.
[0082] In one embodiment, when the processor 302 implements the step of adjusting the model parameters of the image annotation model if the overall confidence level is less than a preset confidence level, and training the model with adjusted parameters based on the medical image set until a target image annotation model with a preset performance gain is obtained, the processor 302 performs the following: adjusting the model parameters of the image annotation model and selecting multiple medical image images from the medical image set as training medical image images; obtaining the target medical annotation result corresponding to each training medical image image and constructing a sample training data based on each training medical image image and the corresponding target medical annotation result to obtain a sample training set, the sample training set including multiple sample training data; adjusting the model parameters of the image annotation model and training the model with adjusted parameters based on the sample training set until a target image annotation model with a preset performance gain is obtained.
[0083] In one embodiment, when the processor 302 selects multiple medical images from the medical image set as training medical image images, it is configured to: select medical image images from the medical image set whose confidence value is less than the overall confidence value as training medical image images, wherein the confidence value is determined by the medical annotation result obtained by the image annotation model for image region division and annotation of the medical image images.
[0084] In one embodiment, when the processor 302 acquires the target medical annotation result corresponding to each of the training medical images, it is configured to: send each of the training medical images to the user so that the user can perform image region division and annotation on the training medical images to obtain the target medical annotation result corresponding to each of the training medical images; and acquire the target medical annotation result corresponding to each of the training medical images sent by the user.
[0085] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the computer device described above can be referred to the corresponding process in the aforementioned image annotation method embodiments, and will not be repeated here.
[0086] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, and the method implemented when the program instructions are executed can refer to various embodiments of the image annotation method of this application.
[0087] The computer-readable storage medium can be an internal storage unit of the computer device described in the foregoing embodiments, such as a hard disk or memory of the computer device. The computer-readable storage medium can be non-volatile or volatile. Alternatively, the computer-readable storage medium can be an external storage device of the computer device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.
[0088] Furthermore, the computer-readable storage medium may primarily include a program storage area and a data storage area, wherein the program storage area may store the operating system, at least one application required for a function, etc.; and the data storage area may store data created based on the use of blockchain nodes, etc.
[0089] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.
[0090] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to include the plural forms.
[0091] It should also be understood that the term "and / or" as used in this specification refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0092] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.
Claims
1. An image annotation method, characterized in that, include: A set of medical images to be labeled and an image labeling model are obtained, wherein the set of images includes at least one medical image; each medical image in the set of images is divided and labeled using the image labeling model to obtain the medical labeling result for each medical image; the feasibility of image labeling is evaluated for the medical labeling results of multiple medical images to obtain the overall credibility. If the overall confidence level is greater than or equal to the preset confidence level, then the medical annotation result of each medical image is taken as the target medical annotation result of each medical image. If the overall credibility is less than the preset credibility, the model parameters of the image annotation model are adjusted, and the model with adjusted parameters is trained according to the medical image set until a target image annotation model with preset performance gain is obtained; according to the target image annotation model, each medical image in the medical image set is divided and annotated to obtain the target medical annotation result for each medical image.
2. The image annotation method as described in claim 1, characterized in that, The image annotation model includes an encoder layer, a modality fusion layer, and an output layer. The step of using the image annotation model to perform image region division and annotation on each medical image in the medical image set to obtain the medical annotation result for each medical image includes: performing feature extraction and encoding processing on each medical image through the encoder layer to obtain image features for each medical image; performing cross-modal interaction on the image features of each medical image through the modality fusion layer to obtain fused features for each medical image; and performing image region division and annotation on the fused features of each medical image through the output layer to obtain the medical annotation result for each medical image.
3. The image annotation method as described in claim 1, characterized in that, The step of evaluating the feasibility of medical annotation of multiple medical images to obtain a comprehensive credibility includes: calculating the credibility of the medical annotation of multiple medical images to obtain a credibility value for each medical image; and averaging the credibility values of all medical images to obtain the comprehensive credibility.
4. The image annotation method as described in claim 3, characterized in that, The step of calculating the credibility of the medical annotation results of multiple medical images to obtain the credibility value of each medical image includes: performing model confidence assessment, interpretability consistency assessment, and cross-modal consistency assessment on the medical annotation results of the medical images to obtain model confidence value, interpretability consistency assessment value, and cross-modal consistency assessment value; and performing a weighted summation of the model confidence value, the interpretability consistency assessment value, and the cross-modal consistency assessment value to obtain the credibility value of the medical image.
5. The image annotation method as described in claim 1, characterized in that, If the overall credibility is less than a preset credibility, the model parameters of the image annotation model are adjusted, and the model with adjusted parameters is trained according to the medical image set until a target image annotation model with a preset performance gain is obtained. This includes: adjusting the model parameters of the image annotation model and selecting multiple medical image images from the medical image set as training medical image images; obtaining the target medical annotation result corresponding to each training medical image image and constructing a sample training data based on each training medical image image and the corresponding target medical annotation result to obtain a sample training set, the sample training set including multiple sample training data; adjusting the model parameters of the image annotation model and training the model with adjusted parameters according to the sample training set until a target image annotation model with a preset performance gain is obtained.
6. The image annotation method as described in claim 5, characterized in that, The step of selecting multiple medical images from the medical image set as training medical image images includes: selecting medical image images from the medical image set whose confidence value is less than the overall confidence value as training medical image images, wherein the confidence value is determined by the medical annotation result obtained by the image annotation model for image region division and annotation of medical image images.
7. The image annotation method as described in claim 5, characterized in that, The step of obtaining the target medical annotation result corresponding to each of the training medical images includes: sending each of the training medical images to the user so that the user can perform image region division and annotation on the training medical images to obtain the target medical annotation result corresponding to each of the training medical images; and obtaining the target medical annotation result corresponding to each of the training medical images sent by the user.
8. An image annotation device, characterized in that, The image annotation device includes an acquisition module, a generation module, a feasibility evaluation module, and a training module, wherein: the acquisition module is used to acquire a set of medical images to be annotated and an image annotation model, the image set including at least one medical image; the generation module is used to perform image region division and annotation on each medical image in the medical image set using the image annotation model, obtaining a medical annotation result for each medical image; the feasibility evaluation module is used to perform image annotation feasibility evaluation on the medical annotation results of multiple medical images, obtaining a comprehensive reliability; the generation module is further used to, if the comprehensive reliability is... If the overall credibility is greater than or equal to a preset credibility, then the medical annotation result of each medical image is used as the target medical annotation result for each medical image. The training module is used to adjust the model parameters of the image annotation model if the overall credibility is less than the preset credibility, and train the model with adjusted parameters according to the medical image set until a target image annotation model with a preset performance gain is obtained. The generation module is also used to perform image region division and annotation on each medical image in the medical image set according to the target image annotation model to obtain the target medical annotation result for each medical image.
9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the steps of the image annotation method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the image annotation method as described in any one of claims 1 to 7.