Liver cancer prediction method combining data enhancement and two-way model training

Abstract

The invention discloses a liver cancer prediction method combining data enhancement and two-way model training, and the method comprises the following steps: S1, inputting a first liver ultrasonic image, extracting a ROI region related to liver cancer, and obtaining a first data set; s2, performing multi-step noise addition and denoising on the first data set to generate a second data set, and fusing the second data set with the first data set to obtain a third data set; s3, constructing a two-way model to perform feature extraction on the third data set, fusing the extracted features to obtain a final image code, inputting the final image code into a classifier, and training the two-way model; and S4, extracting features through the trained two-way model, and performing final prediction of the liver cancer. The data set is enhanced through the diffusion model, dependence on original patient data is reduced, and therefore the problems of medical image data scarcity and privacy protection are solved. In practical application, a synthetic sample can be effectively generated, the privacy of a patient is protected, and meanwhile, the high efficiency and accuracy of model training are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of liver cancer prediction, and in particular to a liver cancer prediction method combining data enhancement and dual-path model training. Background Art

[0002] Hepatocellular Carcinoma (HCC) is one of the most lethal malignant tumors in the world, especially in areas with high incidence of underlying diseases such as hepatitis and cirrhosis, where the morbidity and mortality rates remain high. Although the treatment of liver cancer has been continuously improved in recent years, early detection and metastasis prediction remain major challenges in clinical diagnosis due to the lack of obvious early symptoms of liver cancer and the complex structure of the liver. At present, the diagnosis of liver cancer usually relies on imaging examinations, pathological tests, and the determination of biomarkers. Imaging examinations, such as CT, MRI, and liver ultrasound angiography, have become the main tools for liver cancer screening and diagnosis, especially ultrasound angiography, which has been widely used in the early diagnosis and dynamic monitoring of liver cancer due to its non-invasiveness, low cost, and high sensitivity. By injecting contrast agents to enhance ultrasound images, changes in liver blood flow can be clearly displayed, revealing the blood supply status of the tumor, thereby assisting doctors in discovering and analyzing tumor characteristics.

[0003] Although liver ultrasound angiography has achieved certain results in liver cancer detection, its technical difficulty and diagnostic accuracy are still limited by the operator's experience, image quality, and understanding of the complex structure of the liver. Especially in the detection of early liver cancer and small tumors, the interpretation of liver ultrasound images faces certain limitations, and it is difficult to achieve a comprehensive and accurate diagnosis solely relying on the doctor's subjective judgment due to the limitations of technology and equipment. Existing solutions mainly use deep learning to analyze liver ultrasound images and predict liver cancer. Although these solutions have achieved some results, they still face some technical challenges: First, the noise of the contrast image makes it difficult to segment and extract liver cancer-related areas. Second, due to privacy and cost issues, high-quality contrast image data is scarce and the data diversity is insufficient, which leads to insufficient accuracy and generalization of these methods. Finally, most of the existing solutions are based on single image features and fail to integrate all the information of the liver visceral area and the local information of the tumor area, resulting in insufficient model prediction progress. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a liver cancer prediction method combining data enhancement and dual-path model training.

[0005] The objective of the present invention is achieved through the following technical solutions: The present invention discloses a liver cancer prediction method combining data enhancement and dual-path model training, comprising the following steps: S1, extracting a region of interest from a liver ultrasound image, inputting a first liver ultrasound image, extracting a ROI region related to liver cancer from the first liver ultrasound image, and obtaining a first data set; S2, diffusion model data enhancement, generating a second data set by performing multi-step noise addition and denoising on the first data set, and fusing the second data set with the first data set to obtain a third data set; S3, dual-path model training, constructing a dual-path model to extract features from the third data set, then fusing the extracted features to obtain a final image code, inputting the final image code into a classifier to determine whether the patient has cancer, and training the dual-path model at the same time; S4. Liver cancer prediction, extracts features through the trained two-way model to make the final prediction of liver cancer.

[0006] Preferably, step S1 specifically comprises: segmenting the input first liver ultrasound image through the Medical SAM 2 model , obtain the ROI area related to liver cancer, including the first liver area mask and the first suspected tumor area mask , get the first data set ,in, A label indicating whether the patient to which the image belongs has cancer.

[0007] Preferably, step S2 specifically includes: using the first data set Training the Diffusion Model , and by minimizing the objective loss function Minimize the denoising error, where is the noise image at time t, is the noise image at time t-1, is random noise, The noise predicted by the diffusion model; the second liver ultrasound image is obtained from the noise image through the diffusion model , the second liver ultrasound image First liver ultrasound image The features of the second liver ultrasound image are the same, but include different details and variations; the second liver ultrasound image is processed by Medical SAM 2 segmentation , obtain the second ROI region related to liver cancer, specifically including the second liver region mask and the second suspected tumor area mask , and manually annotated cancer status , get the second data set , the first data set and the second data set are fused to obtain the third data set .

[0008] Preferably, step S3 specifically includes: constructing a dual-path model including a U-Net model and a Transformer model, respectively The liver ultrasound image in the image is processed and the global feature map is obtained by the Transformer model. , U-Net model obtains the third suspected tumor mask After that, the local feature map is obtained by calculating the feature map of this part , and calculate the Dice coefficient loss ,in Represents the mask of the first suspected tumor area and the second suspected tumor area mask The union of For the global feature map and local feature maps Fusion is performed to obtain the final image encoding ; Encode the final image Input into the classifier, the classifier is used to determine whether the patient has cancer, and the loss function The two-way model is trained, where is the coefficient, is the cross entropy loss function, which is calculated as , where N is the number of samples, is the true label i Whether the sample has cancer; This is the prediction value of liver cancer output by the dual-path model.

[0009] Preferably, step S4 specifically includes: after the dual-channel model training is completed, inputting the third liver ultrasound image ,The feature extraction is performed through the trained two-way model to make the final prediction of liver cancer.

[0010] The beneficial effects of the present invention are: 1) Highly accurate prediction of liver cancer. This invention combines the Medical SAM 2 segmentation model and diffusion model data enhancement technology to effectively extract key information from liver ultrasound images and expand the data set. By introducing multi-level image processing and data enhancement, the accuracy of the model in predicting liver cancer is significantly improved, and it can better identify suspected tumor areas in liver images, thereby improving the reliability of the prediction results. In the self-collected liver cancer diagnosis data set, the model accuracy of this patent is improved from 50.34% to 80.12% compared with the baseline model U-Net.

[0011] 2) High robustness of the model. Due to the combination of data enhancement and dual-channel model training, the model can better adapt to different types of liver ultrasound images, thereby improving its robustness to various image noises and variations. In practical applications, the model can process image data from different devices and different shooting conditions, reducing the model's dependence on data deviations and improving its wide applicability.

[0012] 3) Data privacy protection and sustainability. This invention enhances the data set through the diffusion model, reducing the dependence on the original patient data, thereby solving the problem of scarcity and privacy protection of medical imaging data. In practical applications, the combination of diffusion model and generative technology can effectively generate synthetic samples, protect patient privacy, and ensure the efficiency and accuracy of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 The figure is a schematic flow chart of the steps of a liver cancer prediction method combining data enhancement and dual-path model training according to an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0015] The present invention discloses a method for predicting liver cancer based on liver ultrasound images, including four main stages: extraction of regions of interest from liver ultrasound contrast images, diffusion model data enhancement, dual-path model training, and liver cancer prediction. The accuracy of liver cancer prediction is improved by comprehensive processing of liver ultrasound contrast images. First, the liver ultrasound image is segmented, data enhancement is performed in combination with the diffusion model, and feature extraction and fusion are performed using a dual-path model combining U-Net and Transformer, thereby improving the accuracy and reliability of liver cancer prediction and providing more scientific decision support for clinicians. The schematic diagram of the steps is shown in the figure. Figure 1 As shown, the specific steps include: S1. Extraction of key regions of liver ultrasound contrast-enhanced images is the first step of the present invention, and aims to accurately extract ROI regions (region of interest) related to liver cancer from liver ultrasound contrast-enhanced images; Extraction of regions of interest in liver ultrasound images, inputting a first liver ultrasound image, extracting ROI regions related to liver cancer from the first liver ultrasound image, and obtaining a first data set; S2. Liver cancer imaging data is relatively scarce due to privacy and cost issues. The present invention uses diffusion models to perform multi-step noise addition and denoising on the data to generate high-quality images as a supplement to the first data set; diffusion model data enhancement, by performing multi-step noise addition and denoising on the first data set to generate a second data set, and the second data set is fused with the first data set to obtain a third data set; S3, dual-path model training, constructing a dual-path model to extract features from the third data set, then fusing the extracted features to obtain a final image code, inputting the final image code into a classifier to determine whether the patient has cancer, and training the dual-path model at the same time; S4. Liver cancer prediction, extracts features through the trained two-way model to make the final prediction of liver cancer.

[0016] Specifically, step S1 specifically includes: segmenting the input first liver ultrasound image through the Medical SAM 2 model , obtain the ROI area related to liver cancer, including the first liver area mask and the first suspected tumor area mask , get the first data set ,in, A label indicating whether the patient to which the image belongs has cancer.

[0017] Specifically, step S2 specifically includes: using the first data set Training the Diffusion Model , and by minimizing the objective loss function Minimize the denoising error, where is the noise image at time t, is the noise image at time t-1, is random noise, The noise is predicted by the diffusion model; the second liver ultrasound image is obtained by gradually denoising the noise image through the diffusion model. , the second liver ultrasound image First liver ultrasound image The features of the two models are the same, but include different details and variations. In this way, the dataset can be effectively expanded to generate diversified liver ultrasound image data, thereby increasing the amount of training data for the model and improving its ability to recognize liver cancer images; the second liver ultrasound image is processed through Medical SAM 2 segmentation , obtain the second ROI region related to liver cancer, specifically including the second liver region mask and the second suspected tumor area mask , and manually annotated cancer status , get the second data set , the first data set and the second data set are fused to obtain the third data set ,The third dataset contains not only the original image samples but also the ,amplified samples, which makes the dataset richer.

[0018] Specifically, step S3 specifically includes: constructing a dual-path model including a U-Net model and a Transformer model, respectively for the third data set The liver ultrasound image in the image is processed and the global feature map is obtained by the Transformer model. , U-Net model obtains the third suspected tumor mask After that, the local feature map is obtained by calculating the feature map of this part , and calculate the Dice coefficient loss ,in Represents the mask of the first suspected tumor area and the second suspected tumor area mask The union of For the global feature map and local feature maps Fusion is performed to obtain the final image encoding ; Encode the final image Input into the classifier, the classifier is used to determine whether the patient has cancer, and the loss function The two-way model is trained, where is the coefficient, is the cross entropy loss function, which is calculated as , where N is the number of samples, is the true label i Whether the sample has cancer; It is the liver cancer prediction value output by the dual-path model (the value range is 0 to 1).

[0019] Specifically, step S4 specifically includes: after the dual-channel model training is completed, input the third liver ultrasound image , feature extraction is performed through the trained two-way model to make the final prediction of liver cancer. The model outputs the predicted probability of liver cancer, and the patient is judged whether he or she has cancer based on the probability. This step can be deployed in a medical environment to achieve automated early diagnosis of liver cancer and assist decision making.

[0020] Compared with the existing solutions, the key point of the present invention is to combine diffusion model data enhancement and dual-path model training, which significantly improves the accuracy and robustness of liver cancer prediction. Pre-training model to extract the region of interest step: The present invention uses Medical SAM 2 to extract the region of interest of the image to train the dual-path model, ensuring that subsequent training can focus on the key area and improve the prediction performance of the model. Diffusion model data enhancement: The present invention uses a diffusion model for data enhancement, and generates a variety of liver ultrasound image samples through a multi-step noise addition and denoising process to make up for the scarcity of liver cancer imaging data. This process not only expands the data set, but also improves the model's adaptability to different types of image features, thereby improving the accuracy of the prediction. Dual-path model training: The present invention combines the two network architectures of U-Ne and Transformer for feature extraction and feature fusion. U-Net extracts local feature maps, focusing on the detailed information of the tumor area, while Transformer extracts global feature maps, considering the overall information of the liver area. The fusion of the two features provides a more comprehensive image representation, further improving the performance of the model in liver cancer prediction. The pre-trained model extracts the Medical SAM 2 model to segment the liver area and suspected tumor area in the region of interest, and generates the corresponding mask; Diffusion model data enhancement: including the technical details of training the diffusion model for image noise addition and denoising, generating new liver ultrasound image samples, and using this data augmentation method to expand the data set to improve the model's performance in liver cancer prediction tasks. Dual-path model training and feature fusion: including the design and training methods of the U-Net and Transformer dual-path network architecture, especially the technical implementation of how to extract local feature maps through U-Net and fuse them with the global feature maps extracted by Transformer.

[0021] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A liver cancer prediction method combining data enhancement and dual-path model training, characterized in that: The following steps are involved: S1, extracting a region of interest from a liver ultrasound image, inputting a first liver ultrasound image, extracting a ROI region related to liver cancer from the first liver ultrasound image, and obtaining a first data set; S2, diffusion model data enhancement, generating a second data set by performing multi-step noise addition and denoising on the first data set, and fusing the second data set with the first data set to obtain a third data set; S3, dual-path model training, constructing a dual-path model to extract features from the third data set, then fusing the extracted features to obtain a final image code, inputting the final image code into a classifier to determine whether the patient has cancer, and training the dual-path model at the same time; S4. Liver cancer prediction,feature extraction is performed through the trained two-way model to make the final prediction of liver cancer.

2. The method for predicting liver cancer by combining data enhancement and dual-path model training according to claim 1, characterized in that: Step S1 specifically includes: segmenting the first liver ultrasound image input by the Medical SAM 2 model , obtain the ROI area related to liver cancer, including the first liver area mask and the first suspected tumor area mask , get the first data set ,in, A label indicating whether the patient to which the image belongs has cancer.

3. The method for predicting liver cancer by combining data enhancement and dual-path model training according to claim 2, characterized in that: Step S2 specifically includes: using the first data set Training the Diffusion Model , and by minimizing the objective loss function Minimize the denoising error, where is the noise image at time t, is the noise image at time t-1, is random noise, The noise predicted by the diffusion model; the second liver ultrasound image is obtained from the noise image through the diffusion model , the second liver ultrasound image First liver ultrasound image The features of the second liver ultrasound image are the same, but include different details and variations; the second liver ultrasound image is processed by Medical SAM 2 segmentation , obtain the second ROI region related to liver cancer, specifically including the second liver region mask and the second suspected tumor area mask , and manually annotated cancer status , get the second data set , the first data set and the second data set are fused to obtain the third data set .

4. The method for predicting liver cancer by combining data enhancement and dual-path model training according to claim 3, characterized in that: Step S3 specifically includes: constructing a dual-path model including a U-Net model and a Transformer model, respectively The liver ultrasound image in the image is processed and the global feature map is obtained by the Transformer model. , U-Net model obtains the third suspected tumor mask After that, the local feature map is obtained by calculating the feature map of this part , and calculate the Dice coefficient loss ,in Indicates the mask of the first suspected tumor area and the second suspected tumor area mask The union of For the global feature map and local feature maps Fusion is performed to obtain the final image encoding ; Encode the final image Input into the classifier, the classifier is used to determine whether the patient has cancer, and the loss function The two-way model is trained, where is the coefficient, is the cross entropy loss function, which is calculated as , where N is the number of samples, is the true label i Whether the sample has cancer; This is the prediction value of liver cancer output by the dual-path model.

5. The method for predicting liver cancer by combining data enhancement and dual-path model training according to claim 4, characterized in that: Step S4 specifically includes: after the dual-channel model training is completed, the third liver ultrasound image is input ,The feature extraction is performed through the trained two-way model to make the final prediction of liver cancer.

Images