Medical image data annotation generation method, system, equipment and medium

Abstract

The invention provides a medical image data annotation generation method, system and device and a medium, and relates to the technical field of medical image processing, and the method comprises the steps: obtaining a medical image, carrying out the standardization processing of the medical image, and obtaining a standardized medical image; performing target detection and image segmentation on the standardized medical image in sequence by using the trained medical image analysis model to obtain bounding box information and a segmentation mask; generating a binary mask of the standardized medical image according to the segmentation mask; and inputting the standardized medical image, the bounding box information and the binary mask into a trained data annotation generation model, and generating a medical image data annotation by the generation model. According to the method, the specific organ contour or the pathological region shape in the standardized medical image can be accurately marked, the influence of noise and a fuzzy region is effectively overcome, the marking precision is remarkably improved, more comprehensive medical image analysis information is provided, and the generalization ability is high.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and in particular to a method, system, device and medium for generating annotations for medical image data. Background Art

[0002] With the development of medical imaging, medical imaging data is playing an increasingly important role in disease diagnosis and treatment planning. Accurate annotation of medical imaging data is crucial for the training and optimization of medical imaging analysis algorithms, directly impacting algorithm performance and the accuracy of clinical applications. Accurate annotation helps doctors more precisely identify pathological areas and determine disease types, thereby providing more effective treatment for patients.

[0003] Currently, common methods for generating annotations for medical imaging data rely primarily on manual annotation or annotation methods based on machine learning algorithms. Manual annotation methods involve professional medical annotators manually marking organ outlines or pathological areas and their identification information on medical images. While this approach can ensure accuracy to a certain extent due to the expertise of the annotators, it consumes significant manpower and time, is inefficient, and can result in inconsistencies in annotation results due to the annotator's state or subjective differences. Machine learning-based annotation methods, which use algorithms to automatically identify and label target areas in images by learning from existing annotated data, can improve annotation efficiency to a certain extent.

[0004] Labeling methods based on machine learning algorithms often use target detection or image segmentation methods to identify target areas. However, existing labeling methods based on machine learning algorithms can usually only handle a single task in target detection or image segmentation, and have difficulty handling multiple tasks simultaneously, resulting in incomplete labeling results. This makes the results of target area analysis incomplete and inaccurate, and it is difficult to fully reflect the complex relationships in medical images. In addition, lesions in medical images are highly heterogeneous, and the morphology, size, location, and contrast of lesions with surrounding tissues vary widely. When processing complex medical images, existing labeling methods based on machine learning algorithms often have difficulty accurately labeling the outline of organs or the shape of pathological areas. In particular, when there are noisy or blurred areas in the medical images, the labeling accuracy will drop significantly, and the accuracy rate will be low. Summary of the Invention

[0005] In view of the technical problems that the existing annotation methods based on machine learning algorithms can usually only handle single tasks in target detection or image segmentation, and it is difficult to handle multiple tasks at the same time, the results of the target area analysis are not comprehensive and accurate enough, and it is difficult to fully reflect the complex relationships in medical images; it is difficult to accurately mark the outlines of organs or the shapes of pathological areas, and the annotation accuracy is reduced and the accuracy rate is low, the present invention provides a medical image data annotation generation method, system, equipment and medium, which can accurately mark the specific organ outlines or pathological area shapes in standardized medical images, effectively overcome the influence of noise and fuzzy areas, significantly improve the annotation accuracy, provide more comprehensive medical image analysis information, and have strong generalization ability.

[0006] In a first aspect, the present invention provides a method for generating annotations for medical image data, comprising the following steps: S1. Acquire medical images and perform standardization processing on the medical images to obtain standardized medical images. The standardization processing includes resizing, normalization, and denoising. S2. Use the trained medical image analysis model to perform object detection and image segmentation on the standardized medical images. For the object detection task, the medical image analysis model outputs bounding box information for each pathological region or region of interest. The bounding box information includes the bounding box coordinates, the category of each bounding box, and the confidence score. For image segmentation tasks, the medical image analysis model outputs a segmentation mask with the same size as the standardized medical image. The segmentation mask is a binary mask or a multi-category mask. The value of each pixel in the segmentation mask represents the category to which the pixel belongs. S3. Generate a binary mask of the standardized medical image based on the segmentation mask, marking the specific organ contours or pathological area shapes in the standardized medical image; S4. Input the standardized medical image, bounding box information, and binary mask into the trained data annotation generation model, and the data annotation generation model generates medical image data annotations.

[0007] It should be further explained that, in step S2, the medical image analysis model is a deep learning model, and the medical image analysis model includes one or a combination of convolutional neural network (CNN), U-Net, Mask R-CNN, and YOLO.

[0008] In a preferred embodiment, the medical image analysis model is a combination of YOLO and U-Net, using YOLO to perform target detection tasks and using U-Net to perform image segmentation tasks.

[0009] It should be further explained that the steps for building and training the medical image analysis model are as follows: S201. A wide range of medical images, covering both disease samples and normal samples, are collected as training images. These training images are standardized using the same method as in step S1 to obtain standardized training images. A professional physician manually identifies and annotates each standardized training image with a true label. The true label includes the actual bounding box coordinates, bounding box category, and confidence level of each pathological region or region of interest, as well as the specific actual organ outline or pathological region shape. Divide the standardized training medical images into training set, validation set and test set; S202. Build an object detection module for the medical image analysis model using YOLO. The object detection module takes a standardized medical image as input and outputs bounding box information for each pathological region or region of interest, including bounding box coordinates, bounding box category, and confidence level. The image segmentation module of the medical image analysis model is constructed using U-Net. The input of the image segmentation module is a standardized medical image, and the output is a segmentation mask of the same size as the standardized medical image. The segmentation mask is a multi-category mask, and the value of each pixel in the segmentation mask represents the category to which the pixel belongs. The object detection module and the image segmentation module receive the same standardized medical image in parallel, and the outputs of the two modules are fused as the total output of the medical image analysis model; S203. Initialize the medical image analysis model, then input the standardized training medical images in the training set into the medical image analysis model, and output the predicted bounding box information of each pathological region or region of interest and the predicted segmentation mask of the training medical image; S204. Generate a predicted binary mask of the standardized training medical image based on the predicted segmentation mask, and mark the predicted organ contour or pathological region shape in the standardized training medical image; S205. Compare the predicted bounding box information and the annotations of the predicted organ contours or pathological region shapes in each standardized training medical image with the corresponding true labels, calculate the loss value using a loss function, and then update the parameters of the medical image analysis model through backpropagation. During the updating process, the performance of the medical image analysis model is regularly evaluated on the validation set. Hyperparameters are adjusted based on the validation results, and the optimal model parameters are retained to obtain a pre-validated medical image analysis model. S206. Input the standardized medical images in the test set into the pre-validated medical image analysis model, calculate the output results of the object detection mean average approach (mAP) and segmentation Dice coefficient, analyze the cases of misclassification, optimize the model structure through data augmentation, adjusting the anchor box size, or introducing the attention mechanism, retrain and verify the performance, and obtain the trained medical image analysis model.

[0010] It should be further explained that in step S2, the specific method of generating the bounding box is: S211. The medical image analysis model extracts diagnostically relevant image features from standardized medical images. Image features include one or a combination of image texture, shape, area, and density. S212. Determine the pathological region or region of interest based on the image features, and output the bounding box coordinates, bounding box category, and corresponding confidence level for each pathological region or region of interest. Pathological regions include diseased tissue and dysplastic areas, and regions of interest include organs and implants. S213. When multiple bounding boxes enclose the same object, perform non-maximum suppression to remove duplicate bounding boxes. The specific operation is as follows: S2131. Sort bounding boxes that enclose the same object by confidence level. S2132. Starting with the bounding box with the highest confidence, calculate the overlap with other bounding boxes. If the overlap between two bounding boxes exceeds the set duplication threshold, the two bounding boxes are considered duplicates. The bounding box with the lower confidence is removed, and the bounding box with the higher confidence is retained. S214. Output the bounding box coordinates, bounding box categories, and corresponding confidence scores of all retained bounding boxes.

[0011] It should be further explained that step S213 also includes superimposing all retained bounding boxes onto the standardized medical image.

[0012] It should be further noted that the repetition threshold is 0.4.

[0013] It should be further explained that in step S3, when the segmentation mask is a multi-category mask, the specific method of generating a binary mask is: S301. Setting pixel determination threshold according to the bounding box category in step S2; S302. Compare the pixel value of each pixel in the segmentation mask with the pixel judgment threshold. When the pixel value of a pixel is greater than the pixel judgment threshold, set the pixel value of the pixel to 1, and set the pixel values ​​of the remaining pixels to 0.

[0014] It should be further explained that the medical image data annotation includes the diagnostic category of the standardized medical image, and also includes one or a combination of the bounding box coordinates of the pathological area and the binary mask of the standardized medical image.

[0015] In a preferred embodiment, the steps for constructing and training the data annotation generation model are: S401. A wide range of medical images, covering both disease samples and normal samples, are collected as training images. These training images are standardized using the same method as in step S1 to obtain standardized training images. A professional physician manually identifies and annotates each standardized training image with its true label and actual diagnostic category. The true label includes the actual bounding box coordinates, bounding box category, and confidence level for each pathological region or region of interest, as well as the specific actual organ outline or pathological region shape. Divide the standardized training medical images and the corresponding true labels into training set, validation set and test set; S402. Build a data annotation generation model using ResNet. The input of the data annotation generation model is a standardized medical image, bounding box coordinates, bounding box category and confidence, actual organ contour or pathological region shape, and the output is medical image data annotation of the standardized medical image; S403. Initialize the data annotation generation model, then input the training set into the medical image analysis model, and output the predicted medical image data annotation for each standardized training medical image; S404. Compare the predicted diagnostic categories of the standardized training medical images in the medical image data annotations with the actual diagnostic categories of the standardized training medical images, calculate the loss value using a loss function, and then update the parameters of the medical image analysis model through backpropagation. During the updating process, the performance of the medical image analysis model is regularly evaluated on the validation set. Based on the validation results, the hyperparameters are adjusted, and the optimal model parameters are retained to obtain the pre-validation data annotation generation model. S405. Annotate the input data from the test set to generate the model, calculate the classification accuracy, recall rate, and F1 value of the output results, analyze cases of misclassification, optimize the model structure through data augmentation, network structure adjustment, or hyperparameter adjustment, retrain and verify performance, and obtain a trained medical image analysis model.

[0016] It should be further explained that in step S4, the steps of generating medical image data annotations are: S411. Extracting pathological region information from a binary mask of a standardized medical image, the pathological region information including the size, shape, and edge condition of the pathological region; S412. The data annotation generation model classifies the standardized medical image, bounding box information, and pathological region information to obtain the diagnostic category of the standardized medical image; S413. Combine the diagnostic category of the standardized medical image with the bounding box coordinates and / or the binary mask to form a medical image data annotation.

[0017] It should be further explained that the data annotation generation model includes one of ResNet, DenseNet, and EfficientNet.

[0018] In a second aspect, the present invention provides a medical image data annotation generation system for implementing the above-mentioned medical image data annotation generation method, comprising: Medical image import module, used to obtain medical images; A standardization processing module is used to perform standardization processing on medical images to obtain standardized medical images; The medical image analysis module is used to perform target detection and image segmentation on standardized medical images using a trained medical image analysis model. The target detection task outputs the bounding box information of each pathological area or region of interest, and the image segmentation task outputs a segmentation mask with the same size as the standardized medical image. A binary mask generation module is used to generate a binary mask of a standardized medical image based on the segmentation mask, marking the specific organ contour or pathological area shape in the standardized medical image; A data annotation generation module is used to input standardized medical images, bounding box information, and binary masks into a trained data annotation generation model, and the data annotation generation model generates medical image data annotations; The data annotation storage module is used to store medical imaging data annotations.

[0019] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the steps of the above-mentioned medical image data annotation generation method when executing the computer program.

[0020] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned medical image data annotation generation method.

[0021] The beneficial effects of the present invention are: 1. The present invention performs standardization processing on medical images to obtain standardized images. The standardization processing includes resizing, normalization, denoising, and format conversion. This can maximize the preservation of key information in medical images, improve data quality, reduce noise and blurred areas in medical images, facilitate subsequent processing, and improve the accuracy and stability of subsequent medical image data annotation.

[0022] 2. The present invention uses a trained medical image analysis model to perform target detection and image segmentation on standardized medical images in sequence, generates a binary mask based on the image segmentation results, and then inputs the standardized medical image, target detection results and binary mask into the annotation generation model to generate medical image data annotations. By introducing the medical image analysis model and the data annotation generation model, the specific organ contours or pathological area shapes in the standardized medical images can be accurately annotated, significantly improving the annotation accuracy. In particular, when processing complex medical images, it can effectively overcome the influence of noise and blurred areas.

[0023] 3. The medical image analysis model of the present invention can simultaneously handle target detection and image segmentation tasks, generating comprehensive annotation results, including bounding box coordinates, categories, confidence scores, and segmentation masks for generating binary masks, thereby providing more comprehensive medical image analysis information.

[0024] 4. Through standardization and model optimization, the method of the present invention can be applied to various types of medical images, significantly improving the generalization ability of the model and making it more widely applicable in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for the description. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 This is a flowchart of a method for generating annotations for medical image data in one embodiment of the present invention.

[0027] Figure 2 It is a schematic block diagram of a medical image data annotation generation system in one embodiment of the present invention.

[0028] Figure 3 FIG. 1 is a schematic diagram of the hardware structure of an electronic device in one embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the specific embodiments. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] The medical image data annotation generation method involved in this application is mainly aimed at the field of medical image processing technology. The steps include acquiring medical images, performing standardization processing on the medical images, and obtaining standardized medical images; using a trained medical image analysis model to perform target detection and image segmentation on the standardized medical images in sequence to obtain bounding box information and segmentation masks; generating a binary mask of the standardized medical image based on the segmentation mask; inputting the standardized medical image, bounding box information and binary mask into a trained data annotation generation model, and the generation model generates medical image data annotations. Compared with the prior art, the present invention performs standardization processing on medical images to obtain standardized images. The standardization processing includes size adjustment, normalization, denoising and format conversion, which can retain the key information of the medical images to the greatest extent, improve data quality, reduce noise and fuzzy areas in the medical images, facilitate subsequent processing, and improve the accuracy and stability of subsequent medical image data annotations; using a trained medical image analysis model to perform target detection and image segmentation on the standardized medical images in sequence, generating a binary mask based on the image segmentation results, and then inputting the standardized medical image, target detection results and binary mask into the annotation generation model to generate medical image data annotations. By introducing the medical image analysis model The model for generating the model and data annotation can accurately annotate the specific organ contours or pathological area shapes in standardized medical images, significantly improving the annotation accuracy, especially when processing complex medical images, and can effectively overcome the influence of noise and blurred areas; the medical image analysis model can simultaneously handle target detection and image segmentation tasks, and generate comprehensive annotation results, including bounding box coordinates, categories, confidence levels, and segmentation masks for generating binary masks, thereby providing more comprehensive medical image analysis information; through standardized processing and model optimization, the method of the present invention can be applied to various types of medical images, significantly improving the generalization ability of the model, making it more widely applicable in practical applications.

[0031] The medical image data annotation generation method involved in this application is mainly aimed at the technical problems that the annotation methods based on machine learning algorithms in the existing technology can usually only handle a single task in target detection or image segmentation, and it is difficult to handle multiple tasks at the same time. The results of the analysis of the target area are not comprehensive and accurate enough, and it is difficult to fully reflect the complex relationships in medical images; it is difficult to accurately mark the outline of organs or the shape of pathological areas, and the annotation precision is reduced and the accuracy rate is low.

[0032] The following describes in detail the method for generating annotations for medical image data involved in this application. Specific details, such as specific system structures and techniques, are provided for illustrative purposes rather than for limitation, to facilitate a thorough understanding of the embodiments of this application. However, it should be clear to those skilled in the art that this application may also be implemented in other embodiments without these specific details.

[0033] In the medical image data annotation generation method involved in this application, the term "comprising" is used to indicate the presence of the described features, entities, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, entities, steps, operations, elements, components and / or their collections. The terms "including," "comprising," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0034] To facilitate the clear description of the technical solutions of this application, the words "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or order of execution, and the words "first" and "second" do not necessarily mean different.

[0035] The phrases "one embodiment" or "some embodiments" described in this application mean that the specific features, structures, or characteristics described in the embodiment are included in one or more embodiments of the application. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in other embodiments," etc. that appear in different places in this application do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized.

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0037] The medical image data annotation generation method provided in the embodiment of the present invention is executed by a computer device. Accordingly, the medical image data annotation generation system runs in the computer device.

[0038] Figure 1 This is a flowchart of a method for generating annotations for medical image data according to an embodiment of the present invention. Figure 1 The execution subject can be a medical image data annotation generation system. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0039] In the medical image data annotation generation method, Figure 1 As shown, the steps include: Step S1, obtaining a medical image, and performing standardization processing on the medical image to obtain a standardized medical image, wherein the standardization processing includes resizing, normalization, denoising, and converting the format of the medical image into a format that meets the requirements of the medical image analysis model.

[0040] The resizing step can adapt the image to the model input requirements, the normalization step can improve data stability and model convergence speed, the denoising step can reduce interference and improve image quality, and the format conversion step can ensure that the data can be effectively processed by the model, laying a good foundation for subsequent analysis.

[0041] In some specific embodiments, the medical image analysis model includes one or a combination of convolutional neural network (CNN), U-Net, Mask R-CNN, and YOLO.

[0042] Deep learning models or their combinations can fully utilize the advantages of deep learning models to improve the accuracy and efficiency of target detection and image segmentation.

[0043] Step S2: Use the trained medical image analysis model to perform target detection and image segmentation on the standardized medical image in sequence. For the target detection task, the medical image analysis model outputs bounding box information for each pathological region or region of interest. For image segmentation tasks, the medical image analysis model outputs a segmentation mask with the same size as the standardized medical image. The segmentation mask is a binary mask or a multi-category mask, and the value of each pixel in the segmentation mask represents the category to which the pixel belongs.

[0044] Using medical image analysis models for target detection and image segmentation can automatically and accurately locate pathological areas or regions of interest and output bounding box information, while generating segmentation masks, providing accurate basic data for marking specific organ contours or pathological area shapes.

[0045] In some specific embodiments, the steps of building and training a medical image analysis model are: S201. A wide range of medical images, covering both disease samples and normal samples, are collected as training images. These training images are standardized using the same method as in step S1 to obtain standardized training images. A professional physician manually identifies and annotates each standardized training image with a true label. The true label includes the actual bounding box coordinates, bounding box category, and confidence level of each pathological region or region of interest, as well as the specific actual organ outline or pathological region shape. Divide the standardized training medical images into training set, validation set and test set; S202. Build an object detection module for the medical image analysis model using YOLO. The object detection module takes a standardized medical image as input and outputs bounding box information for each pathological region or region of interest, including bounding box coordinates, bounding box category, and confidence level. The image segmentation module of the medical image analysis model is constructed using U-Net. The input of the image segmentation module is a standardized medical image, and the output is a segmentation mask of the same size as the standardized medical image. The segmentation mask is a multi-category mask, and the value of each pixel in the segmentation mask represents the category to which the pixel belongs. The object detection module and the image segmentation module receive the same standardized medical image in parallel, and the outputs of the two modules are fused as the total output of the medical image analysis model; S203. Initialize the medical image analysis model, then input the standardized training medical images in the training set into the medical image analysis model, and output the predicted bounding box information of each pathological region or region of interest and the predicted segmentation mask of the training medical image; S204. Generate a predicted binary mask of the standardized training medical image based on the predicted segmentation mask, and mark the predicted organ contour or pathological region shape in the standardized training medical image; S205. Compare the predicted bounding box information and the annotations of the predicted organ contours or pathological region shapes in each standardized training medical image with the corresponding true labels, calculate the loss value using a loss function, and then update the parameters of the medical image analysis model through backpropagation. During the updating process, the performance of the medical image analysis model is regularly evaluated on the validation set. Hyperparameters are adjusted based on the validation results, and the optimal model parameters are retained to obtain a pre-validated medical image analysis model. S206. Input the standardized medical images in the test set into the pre-validated medical image analysis model, calculate the output results of the object detection mean average approach (mAP) and segmentation Dice coefficient, analyze the cases of misclassification, optimize the model structure through data augmentation, adjusting the anchor box size, or introducing the attention mechanism, retrain and verify the performance, and obtain the trained medical image analysis model.

[0046] The generalization ability of the model can be improved by widely collecting diverse medical images and standardizing their processing; the accuracy of the model can be improved by reasonably dividing the data set for training, verification and testing, and adjusting the hyperparameters based on the verification results; the model is optimized using appropriate loss functions for different tasks; backpropagation is used to update parameters and avoid overfitting, ultimately improving the accuracy and reliability of the model in medical image analysis tasks.

[0047] In some specific embodiments, the specific method of generating a bounding box is: S211. The medical image analysis model extracts diagnostically relevant image features from standardized medical images. Image features include one or a combination of image texture, shape, area, and density. S212. Determine the pathological region or region of interest based on the image features, and output the bounding box coordinates, bounding box category, and corresponding confidence level for each pathological region or region of interest. Pathological regions include diseased tissue and dysplastic areas, and regions of interest include organs and implants. S213. When multiple bounding boxes enclose the same object, perform non-maximum suppression to remove duplicate bounding boxes. The specific operation is as follows: S2131. Sort bounding boxes that enclose the same object by confidence level. S2132. Starting with the bounding box with the highest confidence, calculate the overlap with other bounding boxes. If the overlap between two bounding boxes exceeds the set duplication threshold, the two bounding boxes are considered duplicates. The bounding box with the lower confidence is removed, and the bounding box with the higher confidence is retained. S214. Output the bounding box coordinates, bounding box categories, and corresponding confidence scores of all retained bounding boxes.

[0048] By extracting image features from standardized medical images to determine pathological areas or regions of interest, and then performing bounding box processing and non-maximum suppression, it is possible to accurately locate and identify targets, remove duplicate bounding boxes, improve the accuracy of target detection, and ensure that the bounding box output for the same identified target is accurate and unique.

[0049] In some specific embodiments, step S213 further includes superimposing all retained bounding boxes onto the standardized medical image.

[0050] By superimposing all retained bounding boxes onto standardized medical images, the positions of detected targets in the original images can be visually presented, making it easier for doctors or researchers to observe and analyze them intuitively, and assisting clinical diagnosis and research.

[0051] In some embodiments, the repetition threshold is 0.4.

[0052] Step S3: Generate a binary mask of the standardized medical image based on the segmentation mask, and mark the specific organ contour or pathological area shape in the standardized medical image.

[0053] Generating a binary mask based on the segmentation mask can clearly mark the specific organ contours or pathological area shapes in standardized medical images, making the morphological characteristics of the lesion site or organ more intuitive and facilitating subsequent diagnosis and analysis.

[0054] In some specific embodiments, when the segmentation mask is a multi-category mask, the specific method of generating a binary mask is: S301. Setting pixel determination threshold according to the bounding box category in step S2; S302. Compare the pixel value of each pixel in the segmentation mask with the pixel judgment threshold. When the pixel value of a pixel is greater than the pixel judgment threshold, set the pixel value of the pixel to 1, and set the pixel values ​​of the remaining pixels to 0.

[0055] When the segmentation mask is a multi-category mask, a binary mask is generated by setting the pixel judgment threshold, which can convert complex multi-category information into a concise binary form, highlighting the area of ​​interest and facilitating further analysis and processing of specific areas.

[0056] In step S4, the standardized medical image, bounding box information, and binary mask are input into the trained data annotation generation model, and the data annotation generation model generates medical image data annotations.

[0057] Inputting relevant data into the data annotation generation model generates medical imaging data annotations, which integrates multiple aspects of information to form comprehensive annotations, providing a richer and more accurate basis for medical diagnosis.

[0058] In some specific embodiments, the medical image data annotation includes the diagnostic category of the standardized medical image, and also includes one or a combination of the bounding box coordinates of the pathological region and the binary mask of the standardized medical image.

[0059] Medical image data annotation can provide multi-angle and comprehensive annotation information, helping doctors understand the pathological conditions in the images from different levels and improve the accuracy and comprehensiveness of diagnosis.

[0060] The steps for building and training the data annotation generation model are: S401. A wide range of medical images, covering both disease samples and normal samples, are collected as training images. These training images are standardized using the same method as in step S1 to obtain standardized training images. A professional physician manually identifies and annotates each standardized training image with its true label and actual diagnostic category. The true label includes the actual bounding box coordinates, bounding box category, and confidence level for each pathological region or region of interest, as well as the specific actual organ outline or pathological region shape. Divide the standardized training medical images and the corresponding true labels into training set, validation set and test set; S402. Build a data annotation generation model using ResNet. The input of the data annotation generation model is a standardized medical image, bounding box coordinates, bounding box category and confidence, actual organ contour or pathological region shape, and the output is medical image data annotation of the standardized medical image; S403. Initialize the data annotation generation model, then input the training set into the medical image analysis model, and output the predicted medical image data annotation for each standardized training medical image; S404. Compare the predicted diagnostic categories of the standardized training medical images in the medical image data annotations with the actual diagnostic categories of the standardized training medical images, calculate the loss value using a loss function, and then update the parameters of the medical image analysis model through backpropagation. During the updating process, the performance of the medical image analysis model is regularly evaluated on the validation set. Based on the validation results, the hyperparameters are adjusted, and the optimal model parameters are retained to obtain the pre-validation data annotation generation model. S405. Annotate the input data from the test set to generate the model, calculate the classification accuracy, recall rate, and F1 value of the output results, analyze cases of misclassification, optimize the model structure through data augmentation, network structure adjustment, or hyperparameter adjustment, retrain and verify performance, and obtain a trained medical image analysis model.

[0061] In some specific embodiments, the steps of generating medical image data annotations are: S411. Extracting pathological region information from a binary mask of a standardized medical image, the pathological region information including the size, shape, and edge condition of the pathological region; S412. The data annotation generation model classifies the standardized medical image, bounding box information, and pathological region information to obtain the diagnostic category of the standardized medical image; S413. Combine the diagnostic category of the standardized medical image with the bounding box coordinates and / or the binary mask to form a medical image data annotation.

[0062] Pathological area information is extracted from the binary mask, and then processed by the data annotation generation model to obtain the diagnosis result and combine it with other information to form medical image data annotation. It deeply explores the characteristics of the pathological area, makes the annotation results more clinically valuable, and provides strong support for the accurate diagnosis of the disease and the formulation of treatment plans.

[0063] In some specific embodiments, the data annotation generation model includes one of ResNet, DenseNet, and EfficientNet.

[0064] In a specific embodiment, a method for generating annotations for medical image data includes: Step S1, acquiring a medical image, and performing standardization processing on the medical image to obtain a standardized medical image, wherein the standardization processing includes resizing, normalization, denoising, and converting the format of the medical image into a format that meets the requirements of the medical image analysis model; Step S2: Use the trained medical image analysis model to perform target detection and image segmentation on the standardized medical images in sequence. The medical image analysis model is a combination of YOLO and U-Net. The YOLO model is used to process the target detection task, and the U-Net model is used to process the image segmentation task. For object detection tasks, the medical image analysis model outputs bounding box information for each pathological region or region of interest; The specific method to generate the bounding box is: S211. The medical image analysis model extracts diagnostically relevant image features from standardized medical images. Image features include one or a combination of image texture, shape, area, and density. S212. Determine the pathological region or region of interest based on the image features, and output the bounding box coordinates, bounding box category, and corresponding confidence level for each pathological region or region of interest. Pathological regions include diseased tissue and dysplastic areas, and regions of interest include organs and implants. S213. When multiple bounding boxes enclose the same object, perform non-maximum suppression to remove duplicate bounding boxes. The specific operation is as follows: S2131. Sort bounding boxes that enclose the same object by confidence level. S2132. Starting with the bounding box with the highest confidence, calculate the overlap with other bounding boxes. If the overlap between two bounding boxes exceeds a set duplication threshold, the two bounding boxes are considered duplicates. The bounding box with the lower confidence is removed, and the bounding box with the higher confidence is retained. The duplication threshold is 0.4. S2134. Superimpose all retained bounding boxes onto the standardized medical image; S214. Output the bounding box coordinates, bounding box categories, and corresponding confidence levels of all retained bounding boxes. For image segmentation tasks, the medical image analysis model outputs a segmentation mask with the same size as the standardized medical image. The segmentation mask is a multi-category mask, and the value of each pixel in the segmentation mask represents the category to which the pixel belongs. The steps for building and training a medical image analysis model are: S201. Collect a wide range of medical images (CT, MRI, etc.) covering various disease types (e.g., tumors, inflammation) and normal samples as training images. Standardize these training images using the same method as in step S1 to obtain standardized training images. Professional physicians will manually identify and annotate each standardized training image with a true label, including: The actual bounding box coordinates, bounding box category, and confidence level of each pathological region or region of interest, with a confidence level of 1.0; Specific actual organ contours or shapes of pathological areas; During the labeling process, a method of multi-person labeling + expert review is used to ensure the accuracy of the real labels; The standardized training medical images were divided into training set, validation set and test set in a ratio of 7:1.5:1.5; S202. Use YOLO v5 to construct an object detection module for the medical image analysis model. The object detection module takes a standardized medical image as input and outputs bounding box information for each pathological region or region of interest, including bounding box coordinates, bounding box category, and confidence level. The image segmentation module of the medical image analysis model is constructed using U-Net. The input of the image segmentation module is a standardized medical image, and the output is a segmentation mask of the same size as the standardized medical image. The segmentation mask is a multi-category mask, and the value of each pixel in the segmentation mask represents the category to which the pixel belongs. The object detection module and the image segmentation module receive the same standardized medical image in parallel, and the outputs of the two modules are fused as the total output of the medical image analysis model; S203. Initialize the medical image analysis model, then input the standardized training medical images from the training set into the medical image analysis model, and output the predicted bounding box information of each pathological region or region of interest and the predicted segmentation mask of the training medical image through forward propagation; S204. Generate a predicted binary mask of the standardized training medical image based on the predicted segmentation mask, and mark the predicted organ contour or pathological region shape in the standardized training medical image; S205. Compare the predicted bounding box information and the annotations of the predicted organ contours or pathological region shapes in each standardized training medical image with the corresponding true labels, calculate the loss value using a loss function, and then update the parameters of the medical image analysis model through backpropagation. During the updating process, the performance of the medical image analysis model is regularly evaluated on the validation set. Hyperparameters are adjusted based on the validation results, and the optimal model parameters are retained to obtain a pre-validated medical image analysis model. Among them, the target detection task uses the classification cross entropy loss function to measure the difference between the predicted category and the true category, and the intersection over union (IoU) loss function to evaluate the accuracy of the predicted bounding box information; The image segmentation task uses the Dice loss function to measure the degree of overlap between the predicted organ contour or pathological area shape and the true label; S206. Input the standardized medical images from the test set into the pre-validated medical image analysis model, calculate the output results of object detection mean average approach (mAP) and segmentation Dice coefficient, analyze cases of false positives, optimize the model structure through data augmentation and adjusting anchor box sizes, retrain and verify performance, and obtain a trained medical image analysis model. Step S3, generating a binary mask of the standardized medical image based on the segmentation mask, marking the specific organ contour or pathological area shape in the standardized medical image; The specific method to generate a binary mask is: S301. Setting pixel determination threshold according to the bounding box category in step S2; S302. The pixel value of each pixel in the segmentation mask is compared with the pixel judgment threshold. When the pixel value of a pixel is greater than the pixel judgment threshold, the pixel value of the pixel is set to 1, and the pixel values ​​of the remaining pixels are set to 0; Step S4: input the standardized medical image, bounding box information, and binary mask into the trained data annotation generation model, and the data annotation generation model generates medical image data annotations. The data annotation generation model is ResNet; Among them, medical image data annotation includes the diagnostic category of standardized medical images, the bounding box coordinates of the pathological area and the binary mask of the standardized medical images; The steps for building and training the data annotation generation model are: S401. A wide range of medical images, covering both disease samples and normal samples, are collected as training images. These training images are standardized using the same method as in step S1 to obtain standardized training images. A professional physician manually identifies and annotates each standardized training image with its true label and actual diagnostic category. The true label includes the actual bounding box coordinates, bounding box category, and confidence level for each pathological region or region of interest, as well as the specific actual organ outline or pathological region shape. During the labeling process, a method of multi-person labeling + expert review is used to ensure the accuracy of the real labels; The standardized training medical images and the corresponding true labels are divided into training set, validation set and test set in a ratio of 7:1.5:1.5; S402. Build a data annotation generation model using ResNet. The input of the data annotation generation model is a standardized medical image, bounding box coordinates, bounding box category and confidence, actual organ contour or pathological region shape, and the output is medical image data annotation of the standardized medical image; S403. Initialize the data annotation generation model, then input the training set into the medical image analysis model, and output the predicted medical image data annotation for each standardized training medical image; S404. Compare the predicted diagnostic categories of the standardized training medical images in the medical image data annotations with the actual diagnostic categories of the standardized training medical images, calculate the loss value using a loss function, and then update the parameters of the medical image analysis model through backpropagation. During the updating process, the performance of the medical image analysis model is regularly evaluated on the validation set. Based on the validation results, the hyperparameters are adjusted, and the optimal model parameters are retained to obtain the pre-validation data annotation generation model. Among them, the classification cross entropy loss function is used to measure the difference between the predicted diagnosis category and the true diagnosis category; S405. Analyze the classification accuracy, recall, and F1 score of the output data from the test set. Optimize the model structure by data augmentation, adjusting the network structure, or adjusting hyperparameters. Retrain and verify performance to obtain a trained medical image analysis model. The steps to generate medical imaging data annotations are: S411. Extracting pathological region information from a binary mask of a standardized medical image, the pathological region information including the size, shape, and edge condition of the pathological region; S412. The data annotation generation model classifies the standardized medical image, bounding box information, and pathological region information to obtain the diagnostic category of the standardized medical image; S413. Combine the diagnostic category of the standardized medical image with the bounding box coordinates and / or the binary mask to form a medical image data annotation.

[0065] The following is an embodiment of a medical image data annotation generation system provided by an embodiment of the present disclosure. The medical image data annotation generation system and the medical image data annotation generation methods of the above-mentioned embodiments belong to the same inventive concept. For details not fully described in the embodiment of the medical image data annotation generation system, reference can be made to the embodiments of the above-mentioned medical image data annotation generation method.

[0066] A mobile terminal implementing various embodiments of the present invention will now be described with reference to the accompanying drawings. In the subsequent description, suffixes such as "module," "component," or "unit" used to denote components are used solely to facilitate description of the embodiments of the present invention and do not inherently have specific meanings. Therefore, "module" and "component" may be used interchangeably.

[0067] like Figure 2 As shown, the medical image data annotation generation system includes: Medical image import module, used to obtain medical images; A standardization processing module is used to perform standardization processing on medical images to obtain standardized medical images; The medical image analysis module is used to perform target detection and image segmentation on standardized medical images using a trained medical image analysis model. The target detection task outputs the bounding box information of each pathological area or region of interest, and the image segmentation task outputs a segmentation mask with the same size as the standardized medical image. A binary mask generation module is used to generate a binary mask of a standardized medical image based on the segmentation mask, marking the specific organ contour or pathological area shape in the standardized medical image; A data annotation generation module is used to input standardized medical images, bounding box information, and binary masks into a trained data annotation generation model, and the data annotation generation model generates medical image data annotations; The data annotation storage module is used to store medical imaging data annotations.

[0068] The medical image data annotation generation system of this embodiment is used to implement the medical image data annotation generation method, and the steps include: S1. Acquire medical images and perform standardization processing on the medical images to obtain standardized medical images. Standardization processing includes resizing, normalization, denoising, and converting the medical images into a format that meets the requirements of the medical image analysis model. S2. Use the trained medical image analysis model to perform object detection and image segmentation on the standardized medical images. For the object detection task, the medical image analysis model outputs bounding box information for each pathological region or region of interest. For image segmentation tasks, the medical image analysis model outputs a segmentation mask with the same size as the standardized medical image. The segmentation mask is a binary mask or a multi-category mask. The value of each pixel in the segmentation mask represents the category to which the pixel belongs. S3. Generate a binary mask of the standardized medical image based on the segmentation mask, marking the specific organ contours or pathological area shapes in the standardized medical image; S4. Input the standardized medical image, bounding box information, and binary mask into the trained data annotation generation model, and the data annotation generation model generates medical image data annotations.

[0069] The present application also provides an electronic device for implementing various embodiments of the present invention. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0070] Those skilled in the art will understand that the electronic device structure involved in the embodiments of the present invention does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0071] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0072] The electronic device includes, but is not limited to, components such as a processor and a memory. Those skilled in the art will appreciate that the electronic device structures described in the embodiments of the present invention do not limit the electronic device, and the electronic device may include more or fewer components than shown, or may combine certain components or arrange the components differently.

[0073] In the embodiments of the present invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or claimed herein.

[0074] In the embodiment of the present application, the processor can be implemented by using at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a processor, a controller, a microcontroller, a microprocessor, and an electronic unit designed to perform the functions described herein. In some cases, such an embodiment can be implemented in a controller. For software implementation, an embodiment such as a process or function can be implemented with a separate software module that allows the execution of at least one function or operation. The software code can be implemented by a software application (or program) written in any appropriate programming language, and the software code can be stored in a memory and executed by a controller.

[0075] In addition, the electronic device includes some functional modules not shown, which will not be described here.

[0076] Those skilled in the art will appreciate that various aspects of the electronic device provided herein may be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0077] The present application also provides a storage medium storing a program product capable of implementing the method for generating annotations for medical image data. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product comprising program code that, when executed on a terminal device, causes the terminal device to execute the steps described in the "Exemplary Methods" section above according to various exemplary implementations of the present disclosure.

[0078] The storage medium can be any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0079] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating annotations for medical image data, characterized in that the steps include: S1. Acquire a medical image and perform standardization on the medical image to obtain a standardized medical image, wherein the standardization process includes resizing, normalization, and denoising; S2. Using the trained medical image analysis model, perform target detection and image segmentation on the standardized medical image. For the target detection task, the medical image analysis model outputs bounding box information for each pathological region or region of interest. For the image segmentation task, the medical image analysis model outputs a segmentation mask of the same size as the standardized medical image, wherein the segmentation mask is a binary mask or a multi-category mask, and the value of each pixel in the segmentation mask represents the category to which the pixel belongs; S3. Generate a binary mask of the standardized medical image based on the segmentation mask, marking the specific organ contour or pathological area shape in the standardized medical image; S4. Inputting the standardized medical image, the bounding box information, and the binary mask into a trained data annotation generation model, the data annotation generation model generating medical image data annotations.

2. The method for generating annotations for medical image data according to claim 1, wherein: In step S2, the medical image analysis model includes one or a combination of convolutional neural network, U-Net, Mask R-CNN, and YOLO.

3. The method for generating annotations for medical image data according to claim 1, wherein: In step S2, the specific method of generating the bounding box is: S211. The medical image analysis model extracts diagnosis-related image features from the standardized medical image, wherein the image features include one or a combination of image texture, shape, area, and density; S212. Determine a pathological region or region of interest based on the image features, and output the bounding box coordinates, bounding box category, and corresponding confidence level for each pathological region or region of interest. Pathological regions include diseased tissue and dysplastic areas, and regions of interest include organs and implants. S213. When multiple bounding boxes enclose the same object, perform non-maximum suppression to remove duplicate bounding boxes. The specific operation is as follows: S2131. Sort bounding boxes that enclose the same object by confidence level. S2132. Starting with the bounding box with the highest confidence, calculate the overlap with other bounding boxes. If the overlap between two bounding boxes exceeds the set duplication threshold, the two bounding boxes are considered duplicates. The bounding box with the lower confidence is removed, and the bounding box with the higher confidence is retained. S214. Output the bounding box coordinates, bounding box categories, and corresponding confidence scores of all retained bounding boxes.

4. The method for generating annotations for medical image data according to claim 3, wherein: Step S213 also includes superimposing all retained bounding boxes onto the standardized medical image.

5. The method for generating annotations for medical image data according to claim 3, wherein: In step S3, when the segmentation mask is a multi-category mask, the specific method of generating a binary mask is: S301. Setting pixel determination threshold according to the bounding box category in step S2; S302. Compare the pixel value of each pixel in the segmentation mask with the pixel judgment threshold. When the pixel value of a pixel is greater than the pixel judgment threshold, set the pixel value of the pixel to 1, and set the pixel values ​​of the remaining pixels to 0.

6. The method for generating annotations for medical image data according to claim 1, wherein: The medical image data annotation includes the diagnostic category of the standardized medical image, and also includes one or a combination of the bounding box coordinates of the pathological area and the binary mask of the standardized medical image.

7. The method for generating annotations for medical image data according to claim 6, wherein: In step S4, the steps of generating the medical image data annotation are: S411 extracts pathological region information from the binary mask of the standardized medical image, the pathological region information including the size, shape, and edge of the pathological region; S412. The data annotation generation model classifies the standardized medical image, bounding box information, and pathological region information to obtain a diagnostic category of the standardized medical image; S413. Combine the diagnostic category of the standardized medical image with the bounding box coordinates and / or the binary mask to form the medical image data annotation.

8. A medical image data annotation generation system, characterized in that: A method for generating annotations for medical image data according to any one of claims 1 to 7, comprising: Medical image import module, used to obtain medical images; A standardization processing module is used to perform standardization processing on medical images to obtain standardized medical images; The medical image analysis module is used to perform target detection and image segmentation on standardized medical images using a trained medical image analysis model. The target detection task outputs the bounding box information of each pathological area or region of interest, and the image segmentation task outputs a segmentation mask with the same size as the standardized medical image. A binary mask generation module is used to generate a binary mask of a standardized medical image based on the segmentation mask, marking the specific organ contour or pathological area shape in the standardized medical image; A data annotation generation module is used to input standardized medical images, bounding box information, and binary masks into a trained data annotation generation model, and the data annotation generation model generates medical image data annotations; The data annotation storage module is used to store medical imaging data annotations.

9. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the steps of the medical image data annotation generation method as described in any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that A computer program is stored on the storage medium, and when the computer program is executed by the processor, the steps of the medical image data annotation generation method according to any one of claims 1 to 7 are implemented.

Images