Medical image intelligent labeling and auditing integrated system and method and storage medium

Abstract

The invention discloses an intelligent labeling and auditing integrated system for medical images. The system comprises a task generation and distribution module, an image labeling module, an image auditing module and a database module, different annotation key points are set according to different modes, tasks are distributed to annotation personnel, and the annotation personnel carry out multi-aspect feature annotation on the image; after labeling is completed, a labeling result is sent to an auditor; after the auditor completes auditing, the qualified module data is transmitted to the database module for storage, the annotation data is associated with the original image data for subsequent query, analysis and use, and other data is returned to the annotation personnel or deleted. The problems that a traditional manual labeling mode is long in consumed time and low in efficiency, highly depends on subjective experience of labeling personnel, easily causes uneven labeling quality, and needs to depend on experts for manual auditing one by one, so that the time and labor cost is further increased are solved.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a system, method and storage medium for integrating intelligent annotation and review of medical images. Background Art

[0002] With the rapid development of modern medical technology, medical imaging plays a key role in disease screening, diagnosis, condition monitoring, and treatment effect evaluation. Against the backdrop of increasing demand for intelligent analysis, large-scale data has become the core foundation for driving intelligent development. However, achieving accurate labeling and quality review of large-scale medical imaging data still faces significant challenges. Currently, traditional manual labeling methods are time-consuming, inefficient, and highly dependent on the subjective experience of the labelers. This approach not only easily leads to uneven labeling quality, but also requires reliance on individual manual review by experts, further increasing time and labor costs. Therefore, an integrated system for intelligent labeling and review of medical imaging images plays an indispensable role in optimizing the labeling and review workflow.

[0003] The current typical process involves acquiring images, desensitizing them, classifying them, and converting their formats. Then, professional radiologists, drawing on their medical knowledge and experience, manually draw annotation boxes and add annotation information within image annotation software to label organs, lesions, and other areas. The annotated images are then integrated and transmitted to a team of experienced physicians or reviewers for individual review and approval. This approach is inefficient, lacks a systematic and quantifiable quality assessment mechanism, and is time-consuming.

[0004] There are some solutions at present, but they all have their own shortcomings. The patent with application number CN110993067A discloses a medical image annotation system, including an image acquisition module, a display module and an annotation module. The two-dimensional images of the coronal, sagittal and transverse planes of the three-dimensional medical image are displayed in different areas of the display interface respectively, and the annotation module is used to obtain the plane annotation position of the user on the two-dimensional image of any plane, and the plane annotation position is corresponded to the three-dimensional annotation position obtained in the three-dimensional medical image. The display module simultaneously displays the position information of the three-dimensional annotation position corresponding to each plane on each plane in the display interface. However, the system mainly focuses on the annotation and positioning of the image, does not involve automatic image annotation, and lacks integration with the image review step, and lacks the practicality of efficient image management and collaborative annotation.

[0005] The patent application number CN108461129A provides a medical image annotation method, device and user terminal based on image authentication. The method includes obtaining annotated and unannotated medical image images in the medical image library, and displaying these images as authentication conditions on the login interface, and using the user to obtain the annotation information of the unannotated medical image image based on the target annotation content of the annotated medical image image on the login interface. This patent realizes the annotation of massive medical images through online authentication, improves the annotation efficiency and accuracy, and reduces the annotation cost. However, this method may rely on the user's annotation accuracy, and does not mention how to ensure the consistency and reliability of the user's annotation, and there may be inconsistent annotation quality. In addition, this method mainly focuses on the image annotation process and does not involve the review process of the image annotation results.

[0006] Patent application number CN111671452A discloses a lung CT image annotation system and method for pneumonia. The system includes a database module, a data processing module, a job distribution module, and an image annotation module. The system distributes annotation jobs to designated users through the network. At the same time, the clinical attributes of the annotated CT images are used for deep learning model training to assist users in annotation. However, the system does not include a data review module and does not ensure the consistency and reliability of the annotation. In addition, the method only mentions data of one modality and is not compatible with the annotation of multimodal image data.

[0007] In response to the above problems, this patent proposes an integrated medical image intelligent annotation and review system to solve the above problems. Summary of the Invention

[0008] The purpose of the present invention is to provide an integrated system for intelligent annotation and review of medical images to solve the problem raised in the above-mentioned background technology that the current traditional manual annotation method is time-consuming, inefficient, and highly dependent on the subjective experience of the annotators, which easily leads to uneven annotation quality and requires experts to conduct manual review one by one, thereby further increasing time and labor costs.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a method for integrating intelligent annotation and review of medical images, comprising the following steps: Generate multiple labeling tasks based on the images that need to be labeled; set different labeling priorities for different modalities and assign tasks to labelers, who then label the images in multiple aspects; after labeling is completed, the labeling results are sent to the reviewers; after the review is completed, the "qualified" module data is transferred to the database module for storage, and the labeled data is associated with the original image data for subsequent query, analysis, and use. Other data is returned to the labelers or deleted.

[0010] As an optimal technical solution, a plurality of annotation tasks of a custom number are generated according to the images to be annotated.

[0011] As an optimal technical solution, the task allocation strategy includes two strategies: precise allocation and random allocation. The precise allocation is set according to the user's professional skills, experience level, and workload factors; the random allocation strategy is as follows: if there are m people labeling, the data is evenly distributed into m parts, and users are randomly selected to transmit image data.

[0012] As a preferred technical solution, the marking process includes the following steps: Step S1: Data reception: The task is assigned to the annotator and stored in the "Unannotated" submodule of the "To be Annotated" module in the annotation interface; Step S2: Annotation: The annotator annotates various features of the image, with different annotating focuses for different modalities. The annotating options include: "Abandon Annotation", "Annotated", "Unannotated", and "Questionable". Poor quality images are marked with "Abandon Annotation", and if the annotator is unsure whether the annotation is correct, they are marked with "Questionable". Step S3: Submit for review: When all images are labeled, click the "Submit" option to send the labeling results to the reviewer; if the "Unlabeled" module is not cleared, a prompt box will pop up when you click "Submit": "XXX file is not labeled", and the "Unlabeled" module must be cleared to complete the submission.

[0013] As a preferred technical solution, the image review process includes the following steps: Step S1: Data reception: The images submitted by the annotators are stored in the “Awaiting Review” module of the review interface; Step S2: Review operation: Reviewers can freely choose "All Review" or "Sampling Review" options, and file the images in the "Qualified", "Unqualified", and "Abandon Marking" sub-modules after reviewing them according to the requirements. For "Unqualified" images, demonstration marking can be performed; Step S3: Repair of unqualified images: return the “unqualified” data to the “to be repaired” module of the annotator for modification; Step S4: Audit data storage: After the audit is completed, the "abandon annotation" module data is deleted and the "qualified" module data is transferred to the database module for storage; these annotation data are associated with the original image data for subsequent query, analysis and use.

[0014] A medical image intelligent annotation and review integrated system, further comprising: The task generation and assignment module generates multiple annotation tasks based on the images to be annotated; sets different annotation priorities for different modalities and assigns tasks to annotators; An image annotation module, which receives the annotation task assigned by the task generation and assignment module, annotates the image with various features, and then sends the annotated image to the image review module; An image review module receives the annotated images sent by the image annotation module, transmits the "qualified" image data to the database module for storage after completing the review, and associates the annotated data with the original image data for subsequent query, analysis and use. Other data is returned to the annotator or deleted; Database module, used to store data generated during annotation and review; The image annotation module and image review module have built-in deep convolutional neural networks and recurrent neural networks.

[0015] As a preferred technical solution, it also includes a cloud collaboration module, through which data is transmitted in real time between the labelers and the reviewers, reducing waiting time and speeding up data processing. The labelers can make improvements based on the review opinions at the first time, avoiding the accumulation of erroneous labeling and repetitive work.

[0016] A non-temporary storage medium is used to store a program, which is used to enable a medical image intelligent labeling and review integration system to perform the following actions: executing the above-mentioned medical image intelligent labeling and review integration method.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) Improve work efficiency: An integrated medical image annotation and review system has been built to integrate the collaborative steps of image data processing, storage, distribution, annotation and review, reduce data flow loss between systems, ensure efficient and coherent processes, and significantly reduce time and labor costs.

[0018] (2) Improve annotation consistency: By combining the architecture of deep convolutional neural network and recurrent neural network, image features can be accurately extracted to achieve dynamic target segmentation and intelligent tracking, thereby improving the consistency of annotation results.

[0019] (3) Accelerate the R&D cycle: The system adapts to the diverse image data labeling needs, meets the requirements of intelligent medical image analysis algorithms for high-quality data labels, and facilitates the rapid iteration and clinical transformation of intelligent medical imaging technologies and products. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of the integrated system for intelligent annotation and review of medical images of the present invention; Figure 2 Generate and assign flow charts for the tasks of the present invention; Figure 3 A flowchart of image annotation for the present invention; Figure 4 This is a flowchart of the image review of the present invention. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0022] See also Figure 1 , the present invention provides a technical solution: a medical image intelligent annotation and review integrated system, comprising: a task generation and assignment module, which generates multiple annotation tasks according to the images to be annotated; sets different annotation priorities for different modalities and assigns tasks to annotators; An image annotation module, which receives the annotation task assigned by the task generation and assignment module, annotates the image with various features, and then sends the annotated image to the image review module; An image review module receives the annotated images sent by the image annotation module, transmits the "qualified" image data to the database module for storage after completing the review, and associates the annotated data with the original image data for subsequent query, analysis and use. Other data is returned to the annotator or deleted; Database module, used to store data generated during annotation and review; The image annotation module and image review module have built-in deep convolutional neural networks and recurrent neural networks.

[0023] It also includes a cloud collaboration module, through which data is transmitted in real time between labelers and reviewers, reducing waiting time and speeding up data processing. Labelers can make improvements based on review opinions at the first time, avoiding the accumulation of erroneous labeling and duplication of work.

[0024] The intelligent and integrated labeling method is as follows: 1. Task generation and assignment: ① Task generation: Generate multiple annotation tasks based on the images that need to be annotated (i.e., the pre-processed images stored in the database module). The number of generated tasks can be customized. ② Select task assignment strategy: There are two strategies: precise assignment and random assignment. Precise assignment: Set according to factors such as the user's professional skills, experience level, and workload. For example, complex lesion annotation tasks are assigned to experienced annotators, and simple area annotation tasks are assigned to novices or those in training. Random assignment: If there are m people annotating, the data is evenly distributed into m parts, and users are randomly selected to transmit image data. ③ Implement task assignment: Assign tasks to designated users based on the preset task assignment strategy.

[0025] 2. Image Annotation: ① Data Receiving: The task is assigned to the annotator and stored in the "Unannotated" sub-section of the "To Be Annotated" section of the annotation interface. ② Annotation Operation: The annotator annotates various image features, with different annotation priorities for different modalities. This section includes sub-sections such as "Abandon Annotation," "Annotated," "Unannotated," and "Questionable." Poor image quality allows for abandonment of annotation, while uncertainty about the accuracy of the annotation can be addressed by selecting "Questionable." ③ Submit for Review: Once all images are annotated, click "Submit" to send the annotation results to the reviewer in the corresponding section. If the "Unannotated" section is not cleared, a prompt will pop up when clicking "Submit" (e.g., "XXX file is unannotated; the "Unannotated" section must be cleared before submission).

[0026] 3. Image Review: ① Data Receiving: Images submitted by annotators are stored in the "Awaiting Review" section of the review interface. ② Review Operation: The reviewer can freely select "All Review" or "Sampling Review." After reviewing the images according to the requirements, they are filed in the "Qualified," "Unqualified," and "Discarded" sub-sections. "Unqualified" images can be annotated for demonstration. ③ Unqualified Image Revision: The "Unqualified" data is returned to the annotator's "Awaiting Revision" section for modification. ④ Review Data Storage: After the review is completed, the data in the "Discarded" section is deleted, and the data in the "Qualified" section is transferred to the database module for storage. These annotated data are associated with the original image data for subsequent query, analysis, and use.

[0027] Compared with the existing technology, it has the following innovations: 1. Integrated Medical Image Annotation and Review Process: This patent innovatively integrates data storage, task distribution, image annotation, and image review to achieve a complete image annotation and review workflow. This significantly improves work efficiency, reduces data loss during transfer between different systems or processes, and ensures the consistency and efficiency of the entire process from image acquisition to final review. It can adapt to the comprehensive processing requirements of different types of image data and more comprehensively serve the auxiliary analysis of various disease images in clinical practice and scientific research applications. 2. Cloud-based collaborative operations optimize workflows: This patent implements cloud-based collaborative operations, allowing labelers and reviewers to transmit data in real time, achieving a collaborative workflow in which labeling and reviewing are processed in parallel. In the past, labeling and reviewing were serial, with cumbersome processes and high time costs. Cloud-based collaborative operations break the traditional serial model, allowing the labeling and review processes to proceed synchronously and interact in real time. Labelers can adjust labeling results in a timely manner based on review feedback, and reviewers can also review and provide timely feedback simultaneously, thereby improving overall work efficiency, reducing waiting time, and speeding up the data processing cycle. It also helps to improve the accuracy and consistency of labeling results, promptly discover and correct labeling errors, and provide reliable image analysis data more quickly, accelerating the research and development cycle of intelligent image-assisted analysis software; 3. Artificial intelligence algorithms improve labeling efficiency and accuracy of labeling results: This patent uses an architecture that combines deep convolutional neural networks with recurrent neural networks to extract and analyze image features, achieving intelligent segmentation and tracking of multiple target structures in static images and dynamic videos, and assisting labelers in semi-automatic labeling. Compared with traditional labeling methods that rely solely on manual experience, the method proposed in this patent more accurately locates various tissues, organs, and lesions, and automatically tracks dynamic images. Deep convolutional neural networks are good at processing spatial features of images, while recurrent neural networks can process sequence information (such as time series in dynamic images). The combination of the two effectively improves the accuracy of labeling, while reducing the workload of manual labeling, increasing the labeling speed, and reducing the difference in labeling quality caused by human subjective judgment.

[0028] Based on the above innovations, this patent achieves innovation and breakthroughs in medical image annotation and review by constructing an integrated process for intelligent annotation and review of medical images, using artificial intelligence to improve annotation efficiency and accuracy, and optimizing workflows through cloud-based collaborative operations. This improves work efficiency, reduces labor costs and time consumption, accelerates the R&D cycle of intelligent image-assisted analysis software, and promotes the intelligent development of medical imaging.

[0029] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for integrating intelligent annotation and review of medical images, characterized in that: The following steps are involved: Generate multiple annotation tasks based on the images that need to be annotated; set different annotation priorities for different modalities and assign tasks to annotators, who then annotate the images with various features; After the annotation is completed, the annotation results are sent to the reviewer; after the review is completed, the "qualified" module data is transferred to the database module for storage, and the annotation data is associated with the original image data for subsequent query, analysis and use. Other data are returned to the annotator or deleted; The annotation process includes the following steps: Step S1: Data reception: The task is assigned to the annotator and stored in the "Unannotated" submodule of the "To be Annotated" module in the annotation interface; Step S2: Annotation: The annotator annotates various features of the image, with different annotating priorities for different modalities. The annotating options include: "Abandon Annotation," "Annotated," "Unannotated," and "Questionable." Images of poor quality are marked with "Abandon Annotation," and images unsure of the correctness of the annotation are marked with "Questionable." Step S3: Submit for review: When all images are labeled, click "Submit" to send the results to the reviewer. If the "Unlabeled" block is not cleared, a prompt box will pop up when you click "Submit": "XXX file is not labeled." The "Unlabeled" block must be cleared to complete the submission. The image review process includes the following steps: Step S1: Data reception: The images submitted by the annotators are stored in the "Awaiting Review" module of the review interface; Step S2: Review operation: Reviewers can freely choose "All Review" or "Sampling Review" options. After reviewing the images according to the requirements, they will be filed in the "Pass", "Fail", and "Abandon Marking" sub-modules. For "Failed" images, demonstration marking can be performed; Step S3: Repair of unqualified images: return the "unqualified" data to the "to be repaired" module of the annotator for modification; Step S4: Audit data storage: After the audit is completed, the "abandon annotation" module data is deleted and the "qualified" module data is transferred to the database module for storage; these annotation data are associated with the original image data for subsequent query, analysis and use.

2. The integrated method for intelligent annotation and review of medical images according to claim 1, characterized in that: The method generates a plurality of annotation tasks of a custom number according to the images to be annotated.

3. The integrated method for intelligent annotation and review of medical images according to claim 1, characterized in that: The task allocation strategy includes precise allocation and random allocation. The precise allocation is set according to the user's professional skills, experience level, and workload factors; the random allocation strategy is as follows: if there are m people labeling, the data is evenly distributed into m parts, and users are randomly selected to transmit image data.

4. A medical image intelligent annotation and review integrated system, characterized by: Also included are: The task generation and assignment module generates multiple annotation tasks based on the images to be annotated; sets different annotation priorities for different modalities and assigns tasks to annotators; An image annotation module, which receives the annotation task assigned by the task generation and assignment module, annotates the image with various features, and then sends the annotated image to the image review module; An image review module receives the annotated images sent by the image annotation module, transmits "qualified" image data to the database module for storage after completing the review, and associates the annotated data with the original image data for subsequent query, analysis, and use. Other data is returned to the annotator or deleted; Database module, used to store data generated during annotation and review; The image annotation module and the image review module are built with deep convolutional neural networks and recurrent neural networks.

5. The integrated medical image intelligent annotation and review system according to claim 4 is characterized in that: It also includes a cloud collaboration module, through which data is transmitted in real time between labelers and reviewers, reducing waiting time and speeding up data processing. Labelers can make improvements based on review opinions at the first time, avoiding the accumulation of erroneous labeling and duplication of work.

6. A non-temporary storage medium, characterized in that: It is used to store a program, which is used to enable a medical image intelligent labeling and review integrated system as described in claim 4 or 5 to perform the following actions: execute a medical image intelligent labeling and review integrated method as described in any one of claims 1 to 3 above.

Images