Benign and malignant tumor image classification method and storage medium
Through a generative deep learning model, ordinary ultrasound images are converted into ultrasound contrast images, combined with Gaussian blur and time sequence consistency enhancement, key timing features are extracted, and fully automatic benign and malignant classification of breast tumors is achieved, solving the limitations of traditional ultrasound examinations, and improving the accuracy and popularity of breast cancer screening.
Patent Information
- Application Number
- CN202510270999.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-07
- Publication Date
- 2025-07-01
AI Technical Summary
In the prior art, ultrasound examination lacks effective methods for identifying benign and malignant tumors in breast cancer screening. Traditional ultrasound images rely on morphological characteristics. Ultrasound examination is limited by contrast agent allergies, high costs and professional equipment limitations, making it difficult to widely use in primary hospitals.
Generative deep learning models such as CycleGAN and StableDiffusion are adopted to convert ordinary ultrasound images into ultrasound contrast images through unsupervised learning, combining Gaussian blur and timing consistency enhancement, high-quality ultrasound contrast images are generated, and timing intensity curve features are extracted, and benign and malignant classification is used by machine learning classifiers.
It realizes fully automatic benign and malignant classification of breast tumors, reduces the economic burden of patients, improves diagnostic accuracy and robustness, solves the technical differences between traditional ultrasound and ultrasound imaging, and is suitable for breast cancer screening in primary hospitals.
Smart Images
Figure CN120236121A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of biomedical image technology, and particularly to a method for classifying the benign and malignant of tumor images and a storage medium. Background Art
[0002] The early screening of breast cancer mainly relies on imaging examinations at present, especially ultrasound examination (US). Ordinary ultrasound images mainly rely on morphological features and lack effective reflection of tumor blood perfusion and other deep pathological features, which limits their application in the differentiation between benign and malignant tumors.
[0003] Contrast Enhanced Ultrasound (CEUS) can observe the blood perfusion inside and outside the tumor in real time by injecting contrast agents and using ultrasound technology, significantly improving the diagnostic accuracy of tumors. Especially in the differentiation between benign and malignant tumors, the contrast technology can provide more accurate information. However, the application of CEUS still faces some limitations, mainly including: some patients may have allergic reactions to contrast agents, resulting in the inability to perform contrast examinations; CEUS requires the injection of contrast agents, resulting in higher examination costs, and some patients choose to give up this examination due to economic reasons; CEUS requires special equipment support and professional operators, which limits its wide application in primary hospitals and developing regions.
[0004] Therefore, there is a lack of a method for classifying ordinary ultrasound images of tumors into benign and malignant. Summary of the Invention
[0005] In order to solve the above defects, the present invention proposes a method for classifying the benign and malignant of tumor images and a storage medium.
[0006] The technical solution adopted by the present invention is a method for classifying the benign and malignant of tumor images, and the method includes:
[0007] S100. Input an ordinary ultrasound image;
[0008] S200. Obtain a generative deep learning model, and use the generative deep learning model to generate a contrast-enhanced ultrasound image from the ordinary ultrasound image;
[0009] S300. Analyze the temporal intensity curves of the tumor region and the background region of the contrast-enhanced ultrasound image, and extract key temporal features;
[0010] S400. Classify the benign and malignant of the image according to the key temporal features.
[0011] Further, the generative deep learning model in S200 is a CycleGAN model or a StableDiffusion model.
[0012] Further, the S200 uses the generative deep learning model to generate a contrast-enhanced ultrasound image from the ordinary ultrasound image, which specifically includes:
[0013] S210. Using the generative deep learning model to generate a preliminary contrast-enhanced ultrasound image from the ordinary ultrasound image;
[0014] S220. Performing post-processing operations on the preliminary contrast-enhanced ultrasound image to obtain the contrast-enhanced ultrasound image to be analyzed by the S300;
[0015] The post-processing operation includes introducing a Gaussian blur method, and the Gaussian blur method blurs the edges and transitions of the preliminary contrast-enhanced ultrasound image.
[0016] Further, the post-processing operation further includes introducing a temporal consistency enhancement module, and the temporal consistency enhancement module optimizes the temporal relationship, context information, and motion consistency of the preliminary contrast-enhanced ultrasound image.
[0017] Further, the generative deep learning model in the S200 is obtained through training, and the training strategy includes a progressive training strategy.
[0018] Further, the S300 further includes extracting high-dimensional data from the contrast-enhanced ultrasound image to obtain radiomics features; the S400 further includes classifying the benign and malignant nature of the image according to the radiomics features.
[0019] Further, the S400 further includes: obtaining a machine learning classifier; according to the key temporal features and radiomics features, the machine learning classifier classifies the benign and malignant nature of the image.
[0020] Further, the S100 further includes: performing a preprocessing operation on the ordinary ultrasound image to obtain the ordinary ultrasound image to be processed by the S200;
[0021] The preprocessing operation includes: performing normalization processing on the ordinary ultrasound image; segmenting and extracting the target region of the ordinary ultrasound image.
[0022] Further, after the S200, it further includes: evaluating the quality and authenticity of the contrast-enhanced ultrasound image generated by the S200, and optimizing the parameters of the generative deep learning model according to the evaluation results; the evaluation methods include:
[0023] Quantitative evaluation, which uses the structural similarity index and the perceptual hash value as evaluation indicators, and / or
[0024] Qualitative evaluation, which adopts a double-blind experimental design.
[0025] The present invention also provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned method for classifying the benign and malignant of tumor images is implemented.
[0026] Compared with the prior art, the present invention has the following beneficial effects:
[0027] 1. The method for classifying the benign and malignant of ordinary breast tumor ultrasound images of the present invention generates contrast-enhanced ultrasound images based on existing ordinary ultrasound images, avoiding the use of contrast agents, and at the same time ensuring that the generated images can accurately reflect the blood perfusion characteristics of tumors. Features are extracted according to the generated contrast-enhanced ultrasound images and combined with radiomics features, and finally the automatic classification of the benign and malignant of breast tumor ultrasound images is realized. It reduces the economic burden of patients and solves the technical differences and limitations between traditional ultrasound and contrast-enhanced ultrasound.
[0028] 2. By using CycleGAN for unsupervised learning, the present invention can generate high-quality contrast-enhanced ultrasound images by learning the mapping relationship between ordinary ultrasound images and contrast-enhanced ultrasound images in the absence of high-quality paired data. The unsupervised learning mechanism of CycleGAN makes it perform well in the environment lacking paired data and can effectively avoid the dependence on paired data of traditional methods.
[0029] 3. Through the adversarial training of cycle consistency loss (CycleConsistencyLoss) and generative adversarial network, and adding Gaussian blur to the generator, the subtle differences between ordinary ultrasound images and contrast-enhanced ultrasound images can be effectively reduced, making the style of the generated images closer to the original contrast-enhanced ultrasound images.
[0030] 4. Quantitative and qualitative means are adopted to analyze the effect of this method and further optimize the parameters of the generation model. By generating high-quality contrast-enhanced ultrasound images, the effect of contrast-enhanced ultrasound can be better simulated, ensuring that valuable blood flow information can be provided in tumor diagnosis and meeting clinical needs.
[0031] 5. Bypassing the existing method of manually selecting key frames, a new method of automatically generating time-intensity curves and extracting features is adopted, and finally the accuracy and robustness of breast ultrasound tumor segmentation and benign and malignant classification tasks are improved, providing a more practical solution for clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] The present invention will be described in detail below with reference to the embodiments and the drawings, wherein:
[0033] Figure 1 is a flowchart of the method for classifying the benign and malignant of tumor images in an embodiment;
[0034] Figure 2It is a block diagram of a method for classifying the benign and malignant of tumor images in an embodiment;
[0035] Figure 3 It is a schematic diagram of a general ultrasound image in an embodiment;
[0036] Figure 4 It is a schematic diagram of the image after normalization processing in an embodiment;
[0037] Figure 5 It is a schematic diagram of a black and white binary image in an embodiment;
[0038] Figure 6 It is a schematic diagram of the segmented image in an embodiment;
[0039] Figure 7 It is a schematic diagram of the image after Gaussian blur in an embodiment;
[0040] Figure 8 It is a schematic diagram of the image after temporal consistency enhancement in an embodiment;
[0041] Figure 9 It is the change of the average brightness of the tumor region and the background region with the change of the index frame in an embodiment;
[0042] Figure 10 It is the change of the average brightness of the tumor region and the background region with the change of the index frame in an embodiment;
[0043] Figure 11 It is the change of the average brightness of the tumor region and the background region with the change of the index frame in an embodiment.
[0044] 1. Tumor region; 2. Background region. Detailed implementation manner
[0045] To make the objectives, technical solutions and advantages of the present invention clearer, the following will further describe the embodiments of the present invention in detail with reference to the drawings. The examples of the embodiments are shown in the drawings, in which the same or similar reference numerals represent the same or similar components or components with the same or similar functions from beginning to end. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present invention, and cannot be construed as a limitation to the present invention.
[0046] In order to classify the general ultrasound images of breast tumors as benign or malignant, the present application proposes a method for classifying the benign and malignant of general ultrasound images of breast tumors. This method uses general ultrasound images to generate images with contrast-enhanced ultrasound effects, and further performs temporal intensity curve (TIC) analysis on the generated contrast-enhanced ultrasound images, and finally realizes the automatic classification of the benign and malignant of breast tumor ultrasound images.
[0047] In one embodiment, a method for classifying the benign and malignant nature of tumor images is applicable to the classification of the benign and malignant nature of various tumor images of patients. Refer to Figure 1-2 , the method includes the following steps:
[0048] S100. Input a conventional ultrasound image. Refer to the schematic diagram in Figure 3 . Taking the classification of breast tumor images as an example, first, the conventional ultrasound images of breast cancer patients are input into the entire classification system as initial data. These conventional ultrasound images are obtained by ultrasound examination equipment and mainly contain preliminary information such as the morphology of the tumor. However, due to the lack of effective reflection of key information such as tumor blood perfusion, it is difficult to directly use them for accurate benign and malignant discrimination.
[0049] In a more specific embodiment, S100 further includes: performing a preprocessing operation on the conventional ultrasound image to obtain the conventional ultrasound image to be processed in S200; the preprocessing operation includes: performing a normalization process on the conventional ultrasound image. Refer to the schematic diagram in Figure 4 . The goal of the normalization process is to ensure the consistency of the image in terms of size, gray level range, resolution, etc., so that the model can better process this data. The normalization steps include adjusting the size of the image to ensure that it has the same size ratio, and normalizing the gray values to avoid the influence of differences in different brightness levels in the image on model training.
[0050] Furthermore, the preprocessing operation further includes segmenting and extracting the target region of the conventional ultrasound image. For example, use U-Net to segment the conventional ultrasound image. Refer to the schematic diagram in Figure 5-6 . U-Net is a classic image segmentation network. Its structure includes an encoder and a decoder, and low-level feature information is transmitted through skip connections, which is suitable for segmenting tissue regions with complex details and irregular boundaries. For each conventional ultrasound image, the goal of U-Net is to extract the foreground region of interest, such as lesion regions like tumors. The regions segmented by U-Net will be used as the input for the subsequent generative deep learning model to generate CEUS images. In the black-and-white binary image, the white region marks the region of interest in the image (such as the tumor region), and the black is the background. In other embodiments, Swin-UNet (Swim Transformer U-Net), TransUNet, SegNet, FCN (Fully Convolutional Networks), etc. can also be used for segmenting conventional ultrasound images.
[0051] S200. Obtain a generative deep learning model, and use the generative deep learning model to generate a contrast-enhanced ultrasound image from the conventional ultrasound image.
[0052] In a specific embodiment, the generative deep learning model of S200 is a CycleGAN model. After the region of interest of the ordinary ultrasound image is segmented, the next step is to use the CycleGAN model to convert these ordinary ultrasound images into synthetic CEUS images. CycleGAN is a generative adversarial network (GAN) model for image-to-image conversion. Unlike traditional GANs, CycleGAN does not require paired training data, but is able to learn the mapping relationship between different fields (in this case, ordinary ultrasound images and CEUS images) through adversarial training between two generators and discriminators.
[0053] In the CycleGAN model, the generator is responsible for converting ordinary ultrasound images into synthetic CEUS images, and the discriminator trains the generator by judging the difference between the generated CEUS images and the real CEUS images. In the CycleGAN model, this method designs two sets of generators and discriminators, which are used for the conversion from ordinary ultrasound images to CEUS images and the reverse conversion from CEUS images to ordinary ultrasound images. Through this "cycle consistency" design, it is ensured that the conversion process from ordinary ultrasound images to CEUS images and then to ordinary ultrasound images is consistent without losing key information.
[0054] The discriminator is a classifier that receives an image generated by the generator and predicts whether it is real or synthetic. The generator is trained to generate images of sufficient quality to fool the discriminator into thinking they are real images. At the same time, the discriminator part uses the PatchGAN architecture, which distinguishes synthetic images from real images by judging local areas of the image, further improving the discrimination accuracy.
[0055] Most GAN-based medical image conversion methods require a large amount of paired data for training. However, in the application of ultrasound images and ultrasound contrast images, there is almost no sufficient high-quality cross-domain paired data. Due to issues such as patient personal wishes and the popularity of new-generation ultrasound scanning equipment, there is a serious lack of ultrasound to ultrasound contrast cross-domain paired data that can be used for training in the existing technology.
[0056] The method of this embodiment uses CycleGAN for unsupervised learning, and can generate high-quality ultrasound contrast images by learning the mapping relationship between ordinary ultrasound images and ultrasound contrast images in the absence of high-quality paired data. The unsupervised learning mechanism of CycleGAN enables it to perform well in an environment where paired data is lacking, and can effectively avoid the dependence of traditional methods on paired data.
[0057] In other embodiments, the generative deep learning model of S200 can also be the StableDiffusion model. StableDiffusion is a generative model based on the diffusion process, with good image generation capabilities, especially having advantages in handling details and generating more natural images. In the conversion from ultrasound images to contrast-enhanced ultrasound images, StableDiffusion can be used as an alternative to CycleGAN to further improve the image generation quality. Different from CycleGAN, StableDiffusion generates images through a step-by-step "denoising" process.
[0058] In other embodiments, the generative deep learning model of S200 can also be DenoisingDiffusion Probabilistic Models (DDPM), Guided Diffusion Models, LDM (LatentDiffusion Models), Mamba-Gen, etc.
[0059] In a more specific embodiment, the generative deep learning model in S200 is obtained through training, and the training strategy includes a progressive training strategy. During the training process, in order to further improve the stability of the model and the quality of the generated images, we can adopt a progressive training strategy. Progressive training starts from low-frame-rate images and gradually transitions to high-frame-rate images, avoiding the images generated in the initial stage from being too rough. This strategy helps the model better capture details during the learning process and gradually improve the quality of image generation.
[0060] In a specific embodiment, the use of the generative deep learning model in S200 to generate contrast-enhanced ultrasound images from the ordinary ultrasound images specifically includes:
[0061] S210. Use the generative deep learning model to generate a preliminary contrast-enhanced ultrasound image from the ordinary ultrasound image;
[0062] S220. Perform post-processing operations on the preliminary contrast-enhanced ultrasound image to obtain the contrast-enhanced ultrasound image to be analyzed by S300; the post-processing operations include introducing a Gaussian blur method, and the Gaussian blur method blurs the edges and transitions of the preliminary contrast-enhanced ultrasound image.
[0063] To ensure that the generated CEUS images have a realistic sense, when generating CEUS images, Gaussian blur is added to the output of the generator, especially in the edge regions of the image. CEUS images usually present blurred edges and transitions. Therefore, by simulating this blurred feeling in the generated images, the generated images can look more natural and realistic. See Figure 7Schematic illustration. This process is achieved by adding an adjustable blur layer to the output of the generator, and the intensity of the blur can be adjusted according to the progress of training to ensure the balance of image quality.
[0064] The core formula of Gaussian blur is derived from the two-dimensional Gaussian function, and the formula is as follows:
[0065]
[0066] Where G(x,y) is the two-dimensional Gaussian function; σ is the standard deviation, which controls the intensity of the blur. The larger σ is, the stronger the blur effect; x and y are the offsets from the center of the Gaussian kernel.
[0067] In a more specific embodiment, the post-processing operation further includes introducing a temporal consistency enhancement module, which optimizes the temporal relationship, context information, and motion consistency of the preliminary contrast-enhanced ultrasound images.
[0068] Apply the temporal consistency enhancement module to the generation process from ordinary ultrasound images to contrast-enhanced ultrasound images to ensure that the generated target domain (CEUS) images are consistent with the source domain (US) video frames in terms of temporal, context, and motion consistency. See Figure 8 Schematic illustration. The ultrasound video frames in the source domain usually have continuity, so there are context logical relationships, temporal relationships, and continuous changes brought by the probe movement between frames. These relationships are crucial for the quality and authenticity of the generated images.
[0069] To ensure that the generated contrast-enhanced ultrasound video frames also have these continuity features, the temporal consistency enhancement module (T-CEM) is optimized through the following key steps:
[0070] (a) Optimize the generation process of the current frame by introducing the context information of the previous and next frames. Specifically, the module enhances the relationship between the current frame and the previous and next frames in the time dimension, thereby improving the quality of image generation, ensuring that the generated contrast-enhanced ultrasound images can not only reflect the characteristics of a single frame but also match its context information to form a logically consistent image sequence.
[0071] (b) Ensure a smooth transition in time between the generated contrast-enhanced ultrasound images of each frame and the adjacent previous and next frames. This check evaluates the similarity and change amplitude between image frames to avoid sudden changes or discontinuities in image quality during the generation process. Through this check, the model can identify abnormal changes caused by generation errors and correct them.
[0072] (c) Since the contrast-enhanced ultrasound video is generated based on the dynamic scanning of the probe, the continuous changes caused by the probe movement are included between video frames. The temporal consistency enhancement module ensures that the generated contrast-enhanced ultrasound images can maintain a consistent motion pattern in the time series by modeling these motion features. This modeling not only considers the changes in the image content but also captures the movement of the probe between different frames, thus guaranteeing the motion consistency between the generated contrast-enhanced ultrasound images and the source-domain ultrasound images.
[0073] In summary, the Temporal Consistency Enhancement Module (T-CEM) ensures that the generated contrast-enhanced ultrasound images have higher quality, logic, and continuity by optimizing the temporal relationship, context information, and motion consistency of the generated images. This module provides strong support for the generation of ultrasound videos in clinical applications, especially in the early screening of diseases such as breast tumors, which can greatly improve the diagnostic reliability of the generated images.
[0074] The method for generating contrast-enhanced ultrasound images in this embodiment adopts a post-processing method that combines the CycleGAN model with Gaussian blur and temporal consistency enhancement to convert ordinary ultrasound images into contrast-enhanced ultrasound (CEUS) images, so as to enhance the authenticity and temporal logic of the images. Through the adversarial training of the cycle consistency loss (CycleConsistencyLoss) and the generative adversarial network, and adding Gaussian blur in the generator, the subtle differences between ordinary ultrasound images and contrast-enhanced ultrasound images can be effectively reduced, making the style of the generated images closer to the original contrast-enhanced ultrasound images.
[0075] Regarding the application of Gaussian blur and temporal consistency enhancement, StableDiffusion can also be optimized during the generation process to make the generated contrast-enhanced ultrasound images conform to the realism of medical images through custom training strategies. In addition, the flexibility of StableDiffusion enables it to be customized according to specific needs. For example, different levels of blur processing can be added during the generation process to better simulate the characteristics of ultrasound images.
[0076] S300. Analyze the temporal intensity curves of the tumor region and the background region in the contrast-enhanced ultrasound image, and extract key temporal features.
[0077] In the generated CEUS image with annotations, the annotations are the segmentation results of the tumor region and the background region. On this basis, since continuous video frames are generated, this embodiment further analyzes the temporal intensity curves (TICs) of the tumor region and the background region. The TIC curve reflects the brightness changes of these two regions on the time axis and has significant temporal features. These features are of great significance for judging the benign and malignant nature of tumors.
[0078] By comparing the brightness changes of these two regions on the time axis, key temporal features can be extracted, which have important reference value for judging the benignity and malignancy of breast tumors. Different from traditional methods that rely on simple image processing or empirical judgment, the method of this embodiment can deeply mine image data at the level of temporal features, providing a more objective and accurate tumor feature analysis. Bypassing the existing method of manually selecting key frames, a new method of automatically generating temporal intensity curves and extracting features is adopted, ultimately achieving an improvement in the accuracy and robustness of breast ultrasound tumor segmentation and benignity and malignancy classification tasks, providing a more practical solution for clinical diagnosis.
[0079] S400. Classify the benignity and malignancy of the image according to the key temporal features.
[0080] Combined with clinical analysis, the CEUS brightness of the tumor region is generally higher than that of the background region, which usually indicates rich blood flow and may be a malignant tumor; while when the CEUS brightness of the tumor region is lower than the background, it may be calcification or other breast diseases; if the brightness difference between the tumor region and the background region is not significant, it usually indicates that it may be a benign tumor. By analyzing the wash-in and wash-out characteristics reflected by the TIC curve and the brightness difference between the tumor region and the background region during the contrast agent injection period, this embodiment can effectively extract key temporal features from breast ultrasound images. These features provide a more objective and accurate basis for clinical judgment of the benignity and malignancy of tumors, avoiding the simple processing or empirical judgment of image data by traditional methods. See Figure 9-11 , there are two curves in the figure, which respectively represent the smoothed tumor region 1 and the background region 2. The vertical axis is the average brightness, and the horizontal axis is the index frame. The CEUS-highlighted part represents a high blood flow distribution and is more likely to be a malignant tumor. See Figure 9 ; the dim part represents a low blood flow distribution and is more likely to be calcification or other lesions. See Figure 11 ; being basically the same as the background is more likely to be benign. See Figure 10 .
[0081] In this embodiment, a CycleGAN generation model is proposed to be combined with Gaussian blur method and post-processing method for enhancing temporal consistency to convert ordinary ultrasound images into contrast-enhanced ultrasound (CEUS) images, solving the problem of scarce CEUS image data and improving the realism and temporal logic of the generated images. By analyzing the temporal intensity curves (TIC) of the tumor region and the background region, this embodiment can extract important temporal features reflecting the benignity and malignancy of tumors. Different from traditional methods, this embodiment combines temporal features with clinical analysis methods to objectively and accurately provide a basis for judging the benignity and malignancy of tumors. The TIC analysis technology can further improve the depth and accuracy of breast tumor feature extraction, providing a more reliable diagnostic basis for clinicians.
[0082] In a specific embodiment, the S300 further includes extracting high-dimensional data from the contrast-enhanced ultrasound image to obtain radiomics features; the S400 further includes classifying the malignancy of the image based on the radiomics features.
[0083] Based on the temporal features, this embodiment further combines radiomics features to enhance the accuracy and interpretability of the tumor classification results. Radiomics features can capture subtle changes in the image by extracting high-dimensional data from breast ultrasound images, which are often overlooked in traditional image analysis. By combining temporal features with radiomics features, this embodiment not only improves the accuracy of tumor classification but also enhances the stability and adaptability of the classification model, enabling it to better handle various complex situations in clinical diagnosis.
[0084] Taking the classification of thyroid nodules as an example, by manually annotating the key frames in ultrasound images and contrast-enhanced ultrasound (CEUS) videos, and combining radiomics features, a machine learning classifier (such as support vector machine, random forest, gradient boosting tree, etc.) is used to judge the malignancy. The experiment collected 313 pathologically confirmed thyroid nodules, including 203 malignant nodules and 110 benign nodules. Multiple CEUS key frames were selected and diagnosed through radiomics features and machine learning classifiers. The final results showed that this method was significantly superior to manual diagnosis in terms of accuracy and diagnostic ability.
[0085] This method generates contrast-enhanced ultrasound images based on existing ordinary ultrasound images, avoiding the use of contrast agents, while ensuring that the generated images can accurately reflect the blood perfusion characteristics of tumors. Features are extracted from the generated contrast-enhanced ultrasound images and combined with radiomics features to finally achieve the automatic classification of the malignancy of breast tumor ultrasound images, reducing the economic burden on patients and solving the technical differences and limitations between traditional ultrasound and contrast-enhanced ultrasound.
[0086] In a more specific embodiment, the S400 further includes: obtaining a machine learning classifier; based on the key temporal features and radiomics features, the machine learning classifier classifies the malignancy of the image.
[0087] To complete the task of tumor malignancy classification, this embodiment uses a machine learning classifier (such as support vector machine, random forest, etc.) to train and predict using the extracted features. Through the combination and analysis of these multi-dimensional features, this embodiment provides an efficient, accurate and stable method for classifying the malignancy of breast tumors, which can show good application potential in complex clinical diagnosis environments.
[0088] By automatically segmenting the tumor region and classifying it in combination with radiomics features and ultrasound and CEUS data, the benign and malignant nature of the tumor can be judged more accurately. Compared with the existing traditional method based on manual annotation, this embodiment has significantly improved in terms of automation level and segmentation accuracy, and can provide a more efficient and accurate tumor diagnosis assistance tool for clinicians.
[0089] Based on the temporal features, this embodiment further combines radiomics features and classifies the benign and malignant nature of breast tumors through machine learning classifiers (such as support vector machines, random forests, etc.). Radiomics features can capture the subtle changes in breast ultrasound images, and combining with temporal features can improve the classification accuracy and enhance the stability and adaptability of the classification model. Through this technical point, this embodiment provides an efficient, accurate and stable method for classifying the benign and malignant nature of tumors, which can be applied in complex clinical environments.
[0090] In one embodiment, after S200, it further includes: evaluating the quality and authenticity of the contrast-enhanced ultrasound images generated by S200, and optimizing the parameters of the generative deep learning model according to the evaluation results; the evaluation method includes quantitative evaluation, and the quantitative evaluation uses the structural similarity index and the perceptual hash value as evaluation indicators.
[0091] First, in order to quantitatively evaluate the similarity between the generated CEUS images and the real CEUS images, the structural similarity index (SSIM) and the perceptual hash value (PHV) are used. SSIM is used to measure the similarity between two images in terms of structure, brightness and contrast, and can quantify the similarity between the generated image and the real image; while PHV encodes the image through the perceptual hash algorithm to measure the visual difference between images, especially having strong robustness in terms of the perceptual quality of images. These two evaluation indicators can provide objective numerical support for the quality of the generated images.
[0092] The calculation formula of SSIM is as follows:
[0093]
[0094] Among them, x and y are the pixel values of the two images respectively, μ x and μ y are the average brightness of images x and y respectively, σ x 2 and σ 2 y are the variances of images x and y, and σ xy is the covariance of images x and y, which is used to represent the structural information. The value range of SSIM is from -1 to 1, where 1 means the two images are exactly the same, -1 means exactly the opposite, and 0 means completely dissimilar.
[0095] DCT-based Perceptual Hash Value Calculation Method. Resize the image: First, resize the image to a fixed size, usually an 8x8 or 16x16 image, to reduce the computational amount. Apply the Discrete Cosine Transform (DCT): Perform the discrete cosine transform on each channel of the image to obtain the frequency-domain features. The DCT transforms the spatial-domain features of the image into the frequency domain, enabling the removal of the high-frequency parts (noise) of the image while retaining the main features of the image in the low-frequency parts.
[0096] The calculation formula of DCT is:
[0097]
[0098] where: f(x, y) is the pixel value of the image in the spatial domain; X u,v is the frequency-domain feature of the image after DCT; α(u) and α(v) are constants, usually when u, v > 0, and usually
[0099] Quantize the DCT coefficients: Extract the low-frequency features of the image (such as the first few coefficients of DCT), and then quantize them into binary form. A common approach is to compare the DCT coefficients with their mean value, marking larger values as 1 and smaller values as 0.
[0100] Generate the hash value: Convert the quantized DCT coefficients into a binary hash value, usually converting them into a 16-bit, 64-bit, or 128-bit binary number, which is the perceptual hash value of the image.
[0101] In one embodiment, the evaluation method may also include qualitative evaluation, and the qualitative evaluation adopts a double-blind experimental design. The double-blind experimental design can avoid the possible subjective biases in the evaluation process. Specifically, clinical sonographers compare the generated synthetic CEUS images and real CEUS images without knowing the situation and make diagnostic judgments. This method can effectively avoid the errors caused by factors such as personal experience and preference in the evaluation process, thereby ensuring the objectivity and reliability of the CEUS image generation results.
[0102] Quantitative evaluation includes the Structural Similarity Index (SSIM) and the Perceptual Hash Value (PHV). Qualitative evaluation ensures the reliability and objectivity of the generated images through the visual inspection of clinical sonographers. Quantitative evaluation and qualitative evaluation achieve a comprehensive evaluation of the quality and authenticity of the generated contrast-enhanced ultrasound images. By using quantitative and qualitative means to analyze the effect of this method, the parameters of the generation model can be further optimized. By generating high-quality contrast-enhanced ultrasound images, the effect of contrast-enhanced ultrasound can be better simulated, ensuring that valuable blood flow information can be provided in tumor diagnosis and meeting clinical needs.
[0103] In one embodiment, a storage medium stores a computer program which, when executed by a processor, implements the method for classifying the malignancy of tumor images in the above embodiments.
[0104] In the description of this specification, if terms such as "Embodiment 1", "this embodiment", "in one embodiment", etc. appear, it means that the specific features, structures, materials or characteristics described in connection with this embodiment or example are included in at least one embodiment or example of the invention or the invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example; moreover, the specific features, structures, materials or characteristics described can be combined in a proper manner in any one or more embodiments or examples.
[0105] In the description of this specification, terms such as "connection", "installation", "fixation", "setting", "having", etc. are all understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific situations.
[0106] In the description of this specification, relative terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including the said element.
[0107] The above description of the embodiments is for the convenience of those of ordinary skill in the art to understand and apply the technology of this case. Obviously, those who are familiar with the technology in this field can easily make various modifications to these examples and apply the general principles described herein to other embodiments without creative efforts. Therefore, this case is not limited to the above embodiments, and modifications in the following several situations should be within the protection scope of this case: ① A new technical solution implemented based on the technical solution of the present invention and combined with existing common general knowledge, and the technical effects produced by this new technical solution do not exceed the technical effects of the present invention; ② An equivalent replacement of some features of the technical solution of the present invention using well-known technologies, and the technical effects produced are the same as the technical effects of the present invention; ③ Expansion based on the technical solution of the present invention, and the substantial content of the expanded technical solution does not exceed the technical solution of the present invention; ④ Equivalent transformations made using the content of the specification and drawings of the present invention, directly or indirectly applied in other related technical fields.
Claims
1. A method for classifying benign and malignant tumor images, characterized in that: The method comprises: S100, inputting a common ultrasound image; S200, obtaining a generative deep learning model, and using the generative deep learning model to generate an ultrasound contrast image from the ordinary ultrasound image; S300, analyzing the time series intensity curves of the tumor area and the background area of the ultrasound contrast imaging image to extract key time series features; S400: Classify the image into benign or malignant according to the key time series features.
2. The classification method according to claim 1, characterized in that: The generative deep learning model of S200 is a CycleGAN model or a StableDiffusion model.
3. The classification method according to claim 2, characterized in that: The step S200 of generating an ultrasound contrast-enhanced image from the ordinary ultrasound image using the generative deep learning model specifically includes: S210, generating a preliminary ultrasound contrast image from the ordinary ultrasound image using the generative deep learning model; S220, performing post-processing operations on the preliminary ultrasound contrast image to obtain the ultrasound contrast image to be analyzed in S300; The post-processing operation includes introducing a Gaussian blur method, which blurs the edges and transitions of the preliminary ultrasound contrast image.
4. The classification method according to claim 3, characterized in that: The post-processing operation also includes introducing a temporal consistency enhancement module, which optimizes the temporal relationship, context information and motion consistency of the preliminary ultrasound contrast imaging image.
5. The classification method according to claim 4, characterized in that: The generative deep learning model in S200 is obtained through training, and the training strategy includes a progressive training strategy.
6. The classification method according to any one of claims 1 to 5, characterized in that: The S300 further includes extracting high-dimensional data from the ultrasound contrast-enhanced image to obtain radiomics features; and the S400 further includes classifying the image into benign or malignant according to the radiomics features.
7. The classification method according to claim 6, characterized in that: The S400 also includes: obtaining a machine learning classifier; and classifying the image into benign or malignant according to the key temporal features and radiomics features by the machine learning classifier.
8. The classification method according to any one of claims 1 to 5 or 7, characterized in that: The S100 further includes: performing a preprocessing operation on the common ultrasound image to obtain a common ultrasound image to be processed in the S200; The preprocessing operation includes: performing standardization processing on the common ultrasound image; and segmenting and extracting the target area of the common ultrasound image.
9. The classification method according to any one of claims 1 to 5 or 7, characterized in that: After S200, the method further includes: evaluating the quality and authenticity of the ultrasound contrast imaging image generated in S200, and optimizing the parameters of the generative deep learning model according to the evaluation results; the evaluation method includes: Quantitative evaluation, wherein the quantitative evaluation uses a structural similarity index and a perceptual hash value as evaluation indicators, and / or Qualitative evaluation, the qualitative evaluation adopts a double-blind experimental design.
10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for classifying benign and malignant tumor images according to any one of claims 1 to 9 is implemented.