Novel medical image data enhancement method and system

Abstract

The invention discloses a novel medical image data enhancement method and system, which can fully utilize various characteristics of a medical image by combining image edge information, image text information and image mask information, realize effective fusion of multi-source information, more comprehensively reflect the characteristics of the medical image and improve the image enhancement efficiency. The pre-trained CLIP model is utilized to perform feature embedding, the medical image and the corresponding text description can be mapped to the same high-dimensional space, effective association between the image and the text is realized, the generalization ability of the model to the medical image is improved, and the efficiency of the medical image is improved. By inputting mask image feature embedding and text feature embedding of a specific field or a specific image type into an initial medical image data enhancement diffusion model for optimization, the learning and characterization capability of the model for images in a specific medical field can be further improved, so that the generated medical image not only has higher fidelity, but also has higher resolution. And the lesion information or the anatomical structure in the image can be better reflected.

Description

Technical Field

[0001] The present invention relates to the technical field of image data enhancement, and in particular to a novel medical image data enhancement method and system. Background Art

[0002] With the rapid development of medical technology, medical images play a vital role in disease diagnosis, treatment planning and postoperative evaluation. However, high-quality medical image data is often limited by multiple factors, such as equipment performance, individual differences among patients, imaging conditions, etc., resulting in data scarcity and insufficient diversity. This data limitation not only affects the performance of medical image analysis algorithms, but also hinders the in-depth application of artificial intelligence in the medical field. Therefore, how to effectively enhance medical image data and improve its quantity and quality has become a research hotspot in the current field of medical image processing.

[0003] Traditional medical image data enhancement methods mainly rely on simple image processing techniques, such as rotation, scaling, and flipping. Although these methods can increase the diversity of data to a certain extent, they often lack processing for the specific characteristics of medical images, and it is difficult to effectively improve the model's ability to recognize and analyze complex medical images. Summary of the invention

[0004] In view of this, the present invention proposes a novel medical image data enhancement method and system, which can solve the defect of the prior art that it is difficult to effectively improve the model's recognition and analysis capabilities for complex medical images.

[0005] The technical solution of the present invention is achieved in this way:

[0006] A novel medical image data enhancement method, specifically comprising:

[0007] Obtain medical image datasets and image descriptions;

[0008] Based on the medical image dataset and image description, image edge information, image text information and image mask information are extracted;

[0009] The extracted image edge information, image text information and image mask information are input into the pre-trained CLIP model to obtain edge image feature embedding, text feature embedding and mask image feature embedding that are highly aligned in the feature space;

[0010] The edge image feature embedding, text feature embedding and medical image dataset are input into the pre-trained stable diffusion model for text image to obtain the initial medical image data enhancement diffusion model;

[0011] Inputting the mask image feature embedding and text feature embedding of a specific field or a specific image type into the initial medical image data enhancement diffusion model for model optimization to obtain the final medical image data enhancement diffusion model;

[0012] The specific type of medical image to be processed is input into the corresponding medical image enhancement diffusion model for processing, so as to obtain a large number of similar medical images with rich details, diverse expressions, and authenticity.

[0013] As a further optional solution of the novel medical image data enhancement method, the image edge information is extracted based on a panoramic nested edge detection algorithm, specifically including:

[0014] Preprocessing the medical images in the medical image data set to obtain preprocessed medical images;

[0015] The preprocessed medical image is input into the HED model, which captures edge information at different levels of the image;

[0016] Perform weighted average fusion of edge information at different levels to obtain image edge information;

[0017] Post-process the image edge information and output the post-processed image edge information.

[0018] As a further optional solution of the novel medical image data enhancement method, the process of extracting image text information specifically includes:

[0019] Extract text features from image descriptions;

[0020] Based on OCR technology, the extracted text features are matched with pre-stored character templates to recognize the text in the image, thereby obtaining the image text information.

[0021] As a further optional solution of the novel medical image data enhancement method, the image mask information extraction process specifically includes:

[0022] Use OpenCV library to read medical images from medical image datasets;

[0023] Convert the read color medical image into a grayscale image;

[0024] The grayscale image is binarized according to a preset threshold to obtain image mask information.

[0025] As a further optional solution of the novel medical image data enhancement method, the extracted image edge information, image text information and image mask information are input into a pre-trained CLIP model to obtain edge image feature embedding, text feature embedding and mask image feature embedding that are highly aligned in the feature space, specifically including:

[0026] The image edge information is input into the pre-trained CLIP model, and the image encoder in the pre-trained CLIP model converts the image edge information into a feature vector to obtain edge image feature embedding;

[0027] The image text information is input into the pre-trained CLIP model, and the text encoder in the pre-trained CLIP model converts the text information into a feature vector to obtain text feature embedding;

[0028] The image mask information is input into the pre-trained CLIP model, and the image encoder in the pre-trained CLIP model converts the image mask information into a feature vector to obtain mask image feature embedding.

[0029] As a further optional solution of the novel medical image data enhancement method, the edge image feature embedding, text feature embedding and medical image dataset are input into a pre-trained stable diffusion model for vincetograms for training to obtain an initial medical image data enhancement diffusion model, specifically including:

[0030] Integrate the medical image dataset, edge image feature embedding, and text feature embedding to form a training dataset;

[0031] Use the pre-trained VQGAN model to encode the input medical image dataset into the implicit space to obtain the input data of the pre-trained stable diffusion model;

[0032] The pre-trained stable diffusion model is trained using the implicit encoding of the input medical image dataset, and the integrated marginal image feature embedding and text feature embedding are used to guide the model training process.

[0033] As a further optional solution of the novel medical image data enhancement method, embedding mask image features and text features of a specific field or a specific image type into an initial medical image data enhancement diffusion model for model optimization to obtain a final medical image data enhancement diffusion model specifically includes:

[0034] Integrate medical image datasets of specific fields or specific image types, mask image feature embedding, and text feature embedding to form a training dataset for specific medical images;

[0035] Use the pre-trained VQGAN model to encode the medical image dataset of a specific domain or a specific image type into the implicit space to obtain the input data of the initial stable diffusion model in the specific domain;

[0036] An initial stable diffusion model is trained using implicit encoding of a domain-specific or image-type-specific medical image dataset, and the integrated mask image feature embedding and text feature embedding are used to guide the model training process.

[0037] A novel medical image data enhancement system, comprising:

[0038] An acquisition module, used to acquire medical image datasets and image descriptions;

[0039] An extraction module, used to extract image edge information, image text information and image mask information based on the medical image data set and image description;

[0040] The recognition module is used to input the extracted image edge information, image text information and image mask information into the pre-trained CLIP model to obtain edge image feature embedding, text feature embedding and mask image feature embedding that are highly aligned in the feature space;

[0041] A training module is used to input edge image feature embedding, text feature embedding and medical image data set into a pre-trained stable diffusion model for vince images for training, so as to obtain an initial medical image data enhancement diffusion model;

[0042] An optimization module is used to embed mask image features and text features of a specific field or a specific image type into the initial medical image data enhancement diffusion model for model optimization to obtain a final medical image data enhancement diffusion model;

[0043] The processing module is used to input the specific type of medical image to be processed into the corresponding medical image enhancement diffusion model for processing, so as to obtain a large number of similar medical images with rich details, diverse expressions and authenticity.

[0044] A computing device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned novel medical image data enhancement methods when executing the computer program.

[0045] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any one of the above-mentioned novel medical image data enhancement methods.

[0046] The beneficial effects of the present invention are as follows: by combining image edge information, image text information and image mask information, multiple features of medical images can be fully utilized to achieve effective fusion of multi-source information. This fusion not only enhances the model's understanding of image content, but also improves the accuracy and diversity of generated images. Edge information helps to capture the detailed structure of the image, text information provides a semantic description of the image content, and mask information emphasizes the key areas or lesions in the image. The combination of the three can more comprehensively reflect the characteristics of medical images. Using the pre-trained CLIP model for feature embedding, the rich information learned by the CLIP model on large-scale image-text datasets can be used. Knowledge, mapping medical images and corresponding text descriptions into the same high-dimensional space, achieving effective association between images and texts. This cross-modal feature embedding method helps to improve the model's generalization ability for medical images, enabling it to process more diverse medical image data. By inputting mask image feature embedding and text feature embedding into the initial medical image data enhancement diffusion model for optimization, the model's focus on key areas in medical images can be further refined, and enhanced images that are more in line with medical standards can be generated based on text descriptions. This optimization process ensures that the generated medical images not only have higher fidelity, but also better reflect the lesion information or anatomical structure in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic diagram of a novel medical image data enhancement method of the present invention;

[0049] Figure 2 This is a schematic diagram of the composition of a novel medical image data enhancement system of the present invention. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] refer to Figure 1-2 , a new medical image data enhancement method, specifically including:

[0052] Obtain medical image datasets and image descriptions;

[0053] Based on the medical image dataset and image description, image edge information, image text information and image mask information are extracted;

[0054] The extracted image edge information, image text information and image mask information are input into the pre-trained CLIP model to obtain edge image feature embedding, text feature embedding and mask image feature embedding that are highly aligned in the feature space;

[0055] The edge image feature embedding, text feature embedding and medical image dataset are input into the pre-trained stable diffusion model for text image to obtain the initial medical image data enhancement diffusion model;

[0056] Inputting the mask image feature embedding and text feature embedding of a specific field or a specific image type into the initial medical image data enhancement diffusion model for model optimization to obtain the final medical image data enhancement diffusion model;

[0057] The specific type of medical image to be processed is input into the corresponding medical image enhancement diffusion model for processing, so as to obtain a large number of similar medical images with rich details, diverse expressions, and authenticity.

[0058] In this embodiment, by combining image edge information, image text information and image mask information, it is possible to fully utilize multiple features of medical images and realize effective fusion of multi-source information. This fusion not only enhances the model's understanding of image content, but also improves the accuracy and diversity of generated images. Edge information helps to capture the detailed structure of the image, text information provides a semantic description of the image content, and mask information emphasizes the key areas or lesions in the image. The combination of the three can more comprehensively reflect the characteristics of medical images. Using the pre-trained CLIP model for feature embedding can utilize the rich knowledge learned by the CLIP model on large-scale image-text datasets. , mapping the medical image and the corresponding text description into the same high-dimensional space to achieve effective association between the image and the text. This cross-modal feature embedding method helps to improve the generalization ability of the model for medical images, enabling it to process more diverse medical image data. By inputting the mask image feature embedding and the text feature embedding into the initial medical image data enhancement diffusion model for optimization, the model's focus on key areas in the medical image can be further refined, and enhanced images that are more in line with medical standards can be generated based on the text description. This optimization process makes the generated medical images not only have higher fidelity, but also better reflect the lesion information or anatomical structure in the image.

[0059] Preferably, the image edge information is extracted based on a panoramic nested edge detection algorithm, specifically including:

[0060] Preprocessing the medical images in the medical image data set to obtain preprocessed medical images;

[0061] The preprocessed medical image is input into the HED model, which captures edge information at different levels of the image;

[0062] Perform weighted average fusion of edge information at different levels to obtain image edge information;

[0063] Post-process the image edge information and output the post-processed image edge information.

[0064] In this embodiment, the HED model can capture edge information of different levels in medical images through its unique network structure, which means that it can not only identify significant, coarse-grained edges, but also capture subtle, fine edge structures. This multi-scale detection capability is particularly important for medical images, because medical images often contain structural information of multiple scales, such as blood vessels, cells, tissues, etc.; Since the HED model uses a side output layer (s This high-precision edge location capability helps to retain the key structural information in medical images and provides strong support for subsequent image analysis, diagnosis and other tasks. The quality of medical images has been improved through preprocessing steps (such as denoising, contrast enhancement, etc.), thereby reducing the impact of noise and blur on edge detection. In addition, the HED model also learns robustness to noise and complex backgrounds during training, which makes it more stable and reliable when processing actual medical images. Weighted averaging of edge information at different levels can obtain a more complete and coherent edge information map. This fusion method not only retains the detailed information of edges at all levels, but also reduces edge breaks and false edges, thereby improving the overall quality of edge information. Post-processing of image edge information (such as edge refinement, smoothing, etc.) can further improve the quality of edges and make them more in line with the needs of medical image analysis. The post-processing step can also remove some unnecessary details or noise, making the edge information clearer and more accurate.

[0065] It should be noted that the pre-trained HED model is loaded. The HED model optimizes its performance by fine-tuning on the medical image dataset. The HED model uses the structure of a convolutional neural network (CNN) to extract multi-scale features of the image through multiple convolutional layers, activation layers, and pooling layers. These feature maps capture edge information of the image at different levels at each layer of the network. The model outputs multiple edge prediction maps (corresponding to different levels of the network), which are fused into a final edge map through weighted averaging. The final edge map is post-processed, such as non-maximum suppression (NMS) or edge refinement, to further improve the clarity and accuracy of the edge.

[0066] Preferably, the process of extracting the image text information specifically includes:

[0067] Extract text features from image descriptions;

[0068] Based on OCR technology, the extracted text features are matched with pre-stored character templates to recognize the text in the image, thereby obtaining the image text information.

[0069] In this embodiment, by taking the image description as the starting point for text feature extraction, the method can quickly locate the area in the image that may contain text information, which reduces the full image range that the OCR technology needs to process and improves the efficiency and accuracy of text recognition; the character template matching method based on OCR technology can accurately identify the text in the image by comparing the pre-stored character template with the extracted text features. This method utilizes the prior knowledge of the template and has a certain adaptability to common fonts, font sizes, typesetting methods, etc., thereby improving the recognition accuracy. At the same time, for complex situations such as blur, deformation, occlusion, etc. that may appear in the image, the OCR technology can also make a certain degree of correction and compensation through its built-in algorithm; the entire image text information extraction process is highly automated and can be completed without human intervention, which greatly reduces the workload of medical workers in processing large amounts of medical image data and improves work efficiency. At the same time, automated processing also reduces errors and deviations introduced by human factors and improves the reliability and consistency of data processing.

[0070] It should be noted that there are many methods for feature extraction, and commonly used methods include structure-based feature extraction (extracting text features by analyzing the structural features of text, such as strokes, radicals, etc.) and statistics-based feature extraction (extracting text features by counting the pixel distribution, grayscale value and other features of the text). There are many methods for character recognition, and commonly used methods include template matching (matching the extracted text features with pre-stored character templates to find the most similar character template to recognize the text), neural network method (training the neural network to automatically recognize text in the image) and support vector machine method (training the support vector machine to automatically recognize text in the image).

[0071] Preferably, the process of extracting the image mask information specifically includes:

[0072] Use OpenCV library to read medical images from medical image datasets;

[0073] Convert the read color medical image into a grayscale image;

[0074] The grayscale image is binarized according to a preset threshold to obtain image mask information.

[0075] In this embodiment, the OpenCV library is used to read and convert medical images, and the powerful image processing function of the library can be fully utilized to simplify the extraction process of image mask information. OpenCV provides a wealth of image processing functions, which can efficiently process large-scale medical image data sets and improve processing speed. The grayscale image is binarized according to a preset threshold, and the pixels in the image can be divided into two categories: foreground (target area) and background. This binarization method is simple and effective, and can accurately extract the key area or lesion part in the medical image to form image mask information. The image mask information has important application value in subsequent image analysis, diagnosis and other tasks. By adjusting the preset threshold, the result of image binarization can be flexibly controlled to adapt to the characteristics and needs of different medical images. For example, for medical images with low contrast, the threshold can be appropriately lowered to improve the recognition rate of the target area; for images with noise, a more complex threshold segmentation algorithm may be required to reduce noise interference.

[0076] Preferably, the step of inputting the extracted image edge information, image text information and image mask information into a pre-trained CLIP model to obtain edge image feature embedding, text feature embedding and mask image feature embedding that are highly aligned in feature space specifically includes:

[0077] The image edge information is input into the pre-trained CLIP model. The image encoder in the pre-trained CLIP model converts the image edge information into a feature vector to obtain edge image feature embedding.

[0078] The image text information is input into the pre-trained CLIP model. The text encoder in the pre-trained CLIP model converts the text information into a feature vector to obtain text feature embedding.

[0079] The image mask information is input into the pre-trained CLIP model, and the image encoder in the pre-trained CLIP model converts the image mask information into a feature vector to obtain mask image feature embedding.

[0080] In this embodiment, by inputting image edge information, image text information and image mask information into the CLIP model respectively, the technical solution realizes the multimodal fusion of image and text information. This fusion not only enriches the representation method of image data, but also enables the model to understand the image content more comprehensively, providing richer information support for subsequent tasks; through the feature conversion of the CLIP model, edge image feature embedding, text feature embedding and mask image feature embedding all obtain richer and detailed feature representations. These feature representations not only contain the underlying features of the image or text (such as color, texture, shape, etc.), but also contain high-level semantic information (such as object category, attributes, relationships, etc.). This enhanced feature representation helps subsequent tasks to better understand and utilize this information, thereby improving the performance and accuracy of the task.

[0081] Preferably, the embedding of edge image features, text feature embedding and medical image dataset into a pre-trained stable diffusion model for vincetograms for training to obtain an initial medical image data enhancement diffusion model specifically includes:

[0082] Integrate the medical image dataset, edge image feature embedding, and text feature embedding to form a training dataset;

[0083] Use the pre-trained VQGAN model to encode the input medical image dataset into the implicit space to obtain the input data of the pre-trained stable diffusion model;

[0084] The pre-trained stable diffusion model is trained using the implicit encoding of the input medical image dataset, and the integrated marginal image feature embedding and text feature embedding are used to guide the model training process.

[0085] In this embodiment, the hidden layer code x0 of the input medical image is converted into pure noise through the noise adding process after the input data enhancement diffusion model is input. The noise adding result x0 at any time t is t Only related to the result at time t-1 xt-1 , which belongs to the Markov process, can be expressed as follows:

[0086]

[0087] Among them, ε represents random Gaussian noise, a t =1-β t , β t ∈(0,1) represents the variance coefficient. When T is large enough, due to β t Always greater than 0 and less than 1, will gradually approach 0, then x t will approximately follow an isotropic Gaussian distribution.

[0088] In the denoising process, the data augmentation diffusion model predicts the network μ through the noise θ (x t , t) to perform an approximate reverse process. As shown in the formula, the model starts to generate images from pure noise. Similarly, the denoising result x at any time t-1 is t-1 Only related to the result at time t t .

[0089] p θ (x t-1 |x t )=N(x t-1 ;μ θ (x t ,t),∑ θ (x θ ,t));

[0090] Where N represents Gaussian distribution, θ represents the parameters of the noise prediction network, and μ θ (x t , t) uses the Unet architecture and uses the loss shown in the formula for optimization:

[0091]

[0092] When guiding conditions such as edge image feature embedding (or mask image feature embedding) and text feature embedding are introduced, the loss function will be adjusted as follows:

[0093]

[0094] Where c represents the guide condition. The edge embedding and text embedding in the guide condition will first be adjusted to the same size using a convolution operation, and then added according to the weights w1 and w2 to obtain the final guide condition of the diffusion model. During the training process, the guide condition is compared with the model input x at time t. tThe splicing is passed into the diffusion model. After T time steps, the data enhancement diffusion model generates the final enhanced image x0. x0 not only has diverse representations and rich details, but also contains the basic structure and common characteristics of medical image data.

[0095] In addition, by integrating edge image feature embedding, text feature embedding and medical image datasets, the training dataset not only contains the pixel information of the original image, but also incorporates the edge structure and text description of the image. This fusion of multi-source information makes the training dataset richer and more diverse, which helps the model learn more diverse image representations and generation capabilities. In the field of medical images, this means that more diverse lesion images, anatomical structure images, etc. can be generated, providing doctors with a more comprehensive reference. Adjusting the pre-trained model according to the characteristics of the medical image dataset can make the model better adapt to the particularity of medical image data. Medical images often have complex structures, variable morphologies and subtle lesion features. Through targeted adjustments, the model can capture these features more accurately and generate images that are more in line with medical requirements. The stable diffusion model itself has shown excellent generation capabilities in the text image task and can generate high-quality, high-resolution images. Applying this model to the field of medical image data enhancement and combining it with the guidance of edge and text information can further improve the quality of the generated images. The generated medical images will be clearer and more delicate, which will help doctors make more accurate diagnoses and treatments.

[0096] It should be noted that, according to the characteristics of the medical image dataset, the pre-trained stable diffusion model for Vincent map is adjusted. Specifically, the input layer parameters of the model are adjusted according to the size, format and number of channels of the medical image data. If necessary, preprocessing steps such as image normalization, cropping, scaling, etc. can be added to ensure the consistency of the input data. In view of the complexity of medical images, it may be necessary to optimize the feature extractor part of the model to enhance its ability to extract medical image features. This may include increasing the number of convolutional layers, changing the size and step size of the convolution kernel, introducing an attention mechanism, etc. A large amount of medical image data is used for training to cover different anatomical structures, pathological types and imaging methods. During the training process, regularization techniques (such as Dropout, L2 regularization, etc.) can be introduced to prevent overfitting. At the same time, data augmentation techniques (such as rotation, flipping, scaling, etc.) can be used to increase the diversity of training data. The adjusted model is evaluated using an independent test dataset to verify its performance and accuracy in generating medical images, and the model is further optimized and adjusted based on the evaluation results.

[0097] Preferably, embedding the mask image features and text features of a specific field or a specific image type into an initial medical image data enhancement diffusion model for model optimization to obtain a final medical image data enhancement diffusion model specifically includes:

[0098] Integrate medical image datasets of specific fields or specific image types, mask image feature embedding, and text feature embedding to form a training dataset for specific medical images;

[0099] Use the pre-trained VQGAN model to encode the medical image dataset of a specific domain or a specific image type into the implicit space to obtain the input data of the initial stable diffusion model in the specific domain;

[0100] An initial stable diffusion model is trained using implicit encoding of a domain-specific or image-type-specific medical image dataset, and the integrated mask image feature embedding and text feature embedding are used to guide the model training process.

[0101] In this embodiment, by integrating the mask image feature embedding and text feature embedding of a specific type of medical image into the same size and performing weighted sum as a guiding condition input, this design enables the model to comprehensively consider the local details of the image (provided by the mask image feature embedding) and the global semantic information (provided by the text feature embedding) when generating an image, thereby achieving more refined and accurate image generation. This enhanced feature fusion capability helps to improve the quality and diversity of the generated images. On the basis of the initial model, by introducing the mask image feature embedding and the text feature embedding for model optimization, the model can be made more adaptable to the particularity and complexity of medical image data. Medical images often contain rich anatomical structure and lesion information, and the mask image feature embedding It can highlight these key areas, and text feature embedding provides descriptive information about the image content. This combination enables the model to more accurately capture the features of medical images and generate images that meet medical requirements. Due to the introduction of mask image feature embedding and text feature embedding, the model can be subject to more constraints and guidance when generating images. Mask image feature embedding can help the model more accurately locate key areas in the image and generate more refined local details. Text feature embedding can provide global semantic guidance for the model to ensure that the generated image is consistent with the given text description. This dual constraint helps to improve the accuracy of the generated image. The model's ability to generate specific types of medical data has been significantly enhanced in terms of targeting and professionalism. This means that the images generated by the model not only have the basic characteristics of medical images, such as high definition and precise tissue structure expression, but also deeply integrate the unique features and details of specific types of data, which is of great significance for promoting the development of medical image analysis, diagnosis and teaching.

[0102] A novel medical image data enhancement system, comprising:

[0103] An acquisition module, used to acquire medical image datasets and image descriptions;

[0104] An extraction module, used to extract image edge information, image text information and image mask information based on the medical image data set and image description;

[0105] The recognition module is used to input the extracted image edge information, image text information and image mask information into the pre-trained CLIP model to obtain edge image feature embedding, text feature embedding and mask image feature embedding that are highly aligned in the feature space;

[0106] A training module is used to input edge image feature embedding, text feature embedding and medical image data set into a pre-trained stable diffusion model for vince images for training, so as to obtain an initial medical image data enhancement diffusion model;

[0107] An optimization module is used to embed mask image features and text features of a specific field or a specific image type into the initial medical image data enhancement diffusion model for model optimization to obtain a final medical image data enhancement diffusion model;

[0108] The processing module is used to input the specific type of medical image to be processed into the corresponding medical image enhancement diffusion model for processing, so as to obtain a large number of similar medical images with rich details, diverse expressions and authenticity.

[0109] A computing device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned novel medical image data enhancement methods when executing the computer program.

[0110] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any one of the above-mentioned novel medical image data enhancement methods.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A novel medical image data enhancement method, characterized in that: Specifically include: Obtain medical image datasets and image descriptions; Based on the medical image dataset and image description, image edge information, image text information and image mask information are extracted; The extracted image edge information, image text information and image mask information are input into the pre-trained CLIP model to obtain edge image feature embedding, text feature embedding and mask image feature embedding that are highly aligned in the feature space; The edge image feature embedding, text feature embedding and medical image dataset are input into the pre-trained stable diffusion model for text image to obtain the initial medical image data enhancement diffusion model; Inputting the mask image feature embedding and text feature embedding of a specific field or a specific image type into the initial medical image data enhancement diffusion model for model optimization to obtain the final medical image data enhancement diffusion model; The specific type of medical image to be processed is input into the corresponding medical image enhancement diffusion model for processing, so as to obtain a large number of similar medical images with rich details, diverse expressions, and authenticity.

2. A novel medical image data enhancement method according to claim 1, characterized in that: The image edge information is extracted based on a panoramic nested edge detection algorithm, specifically including: Preprocessing the medical images in the medical image data set to obtain preprocessed medical images; The preprocessed medical image is input into the HED model, which captures edge information at different levels of the image; Perform weighted average fusion of edge information at different levels to obtain image edge information; Post-process the image edge information and output the post-processed image edge information.

3. A novel medical image data enhancement method according to claim 2, characterized in that: The process of extracting the image text information specifically includes: Extract text features from image descriptions; Based on OCR technology, the extracted text features are matched with pre-stored character templates to recognize the text in the image, thereby obtaining the image text information.

4. A novel medical image data enhancement method according to claim 3, characterized in that: The process of extracting the image mask information specifically includes: Use OpenCV library to read medical images from medical image datasets; Convert the read color medical image into a grayscale image; The grayscale image is binarized according to a preset threshold to obtain image mask information.

5. A novel medical image data enhancement method according to claim 4, characterized in that: The extracted image edge information, image text information and image mask information are input into the pre-trained CLIP model to obtain edge image feature embedding, text feature embedding and mask image feature embedding that are highly aligned in the feature space, specifically including: The image edge information is input into the pre-trained CLIP model. The image encoder in the pre-trained CLIP model converts the image edge information into a feature vector to obtain edge image feature embedding. The image text information is input into the pre-trained CLIP model. The text encoder in the pre-trained CLIP model converts the text information into a feature vector to obtain text feature embedding. The image mask information is input into the pre-trained CLIP model, and the image encoder in the pre-trained CLIP model converts the image mask information into a feature vector to obtain mask image feature embedding.

6. A novel medical image data enhancement method according to claim 5, characterized in that: The edge image feature embedding, text feature embedding and medical image data set are input into a pre-trained stable diffusion model for vincetograms for training to obtain an initial medical image data enhancement diffusion model, specifically including: Integrate the medical image dataset, edge image feature embedding, and text feature embedding to form a training dataset; Use the pre-trained VQGAN model to encode the input medical image dataset into the implicit space to obtain the input data of the pre-trained stable diffusion model; The pre-trained stable diffusion model is trained using the implicit encoding of the input medical image dataset, and the integrated marginal image feature embedding and text feature embedding are used to guide the model training process.

7. A novel medical image data enhancement method according to claim 6, characterized in that: The embedding of mask image features and text features of a specific field or a specific image type into the initial medical image data enhancement diffusion model for model optimization to obtain the final medical image data enhancement diffusion model specifically includes: Integrate medical image datasets of specific fields or specific image types, mask image feature embedding, and text feature embedding to form a training dataset for specific medical images; Use the pre-trained VQGAN model to encode the medical image dataset of a specific domain or a specific image type into the implicit space to obtain the input data of the initial stable diffusion model in the specific domain; An initial stable diffusion model is trained using implicit encoding of a domain-specific or image-type-specific medical image dataset, and the integrated mask image feature embedding and text feature embedding are used to guide the model training process.

8. A novel medical image data enhancement system, characterized in that: include: An acquisition module, used to acquire medical image datasets and image descriptions; An extraction module, used to extract image edge information, image text information and image mask information based on the medical image data set and image description; The recognition module is used to input the extracted image edge information, image text information and image mask information into the pre-trained CLIP model to obtain edge image feature embedding, text feature embedding and mask image feature embedding that are highly aligned in the feature space; A training module is used to input edge image feature embedding, text feature embedding and medical image data set into a pre-trained stable diffusion model for vince images for training, so as to obtain an initial medical image data enhancement diffusion model; An optimization module is used to embed mask image features and text features of a specific field or a specific image type into the initial medical image data enhancement diffusion model for model optimization to obtain a final medical image data enhancement diffusion model; The processing module is used to input the specific type of medical image to be processed into the corresponding medical image enhancement diffusion model for processing, so as to obtain a large number of similar medical images with rich details, diverse expressions and authenticity.

9. A computing device, characterized in that The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the novel medical image data enhancement method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the novel medical image data enhancement method according to any one of claims 1 to 7 are implemented.

Images