A method and system for the adjunctive screening of hepatocellular carcinoma

Abstract

The application discloses a hepatocellular carcinoma auxiliary screening method and system. The method comprises the following steps: inputting a liver occupying gray scale ultrasonic image of a to-be-screened object into a trained hepatocellular carcinoma (HCC) diagnosis model for screening to obtain an HCC screening result of the object; the HCC diagnosis model is obtained by training a neural network model by using training image samples generated by the liver occupying gray scale ultrasonic image; and the liver occupying gray scale ultrasonic image at least comprises a liver occupying gray scale ultrasonic image of which serum alpha-fetal protein (AFP) detection is negative and pathological diagnosis is hepatocellular carcinoma. The diagnosis model obtained by the technical scheme provided by the application can accurately identify patients with serum alpha-fetal protein negative and hepatocellular carcinoma, and has low cost, thereby providing a reference for medical staff to judge whether fine examination needs to be performed on patients with serum alpha-fetal protein negative and hepatocellular carcinoma risk.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent medical treatment, in particular to a liver cancer auxiliary screening method and system based on a machine vision neural network model. BACKGROUND

[0002] Liver cancer ranks fourth in the world in terms of cancer-related deaths. Among them, hepatocellular carcinoma (HCC) is the most common primary malignant tumor of the liver, accounting for about 85%-90% of all primary liver cancers. In China, nearly 80% of HCC cases are related to hepatitis B virus (HBV). For high-risk groups of HCC, major medical guidelines around the world recommend serum alpha-fetoprotein (AFP) and liver ultrasound as screening methods (APASL 2017, EASL 2018, AASLD 2018, JSH 2014, China 2019). Among HCC high-risk patients, liver space-occupying lesions found by ultrasound gray-scale images and accompanied by serum AFP elevation can easily screen out HCC cases. However, whether the tumor is in the early stage or the terminal stage, only about one-third of HCC patients can detect AFP elevation, and AFP-negative HCC cases still account for a large part of HCC cases. Therefore, for patients with AFP-negative and HCC risk, a cost-effective, fast and reliable screening method is needed.

[0003] Currently, conventional ultrasound (US) has become a widely used liver tumor screening tool due to its advantages of being fast, non-invasive and cost-effective. However, gray-scale ultrasound images have low accuracy in differentiating AFP-negative HCC and benign liver lesions (FLL), and liver space-occupying lesions found in high-risk cases usually require contrast-enhanced imaging (MRI / CT / US) or even invasive fine examination such as biopsy. Obviously, fine examination is not suitable for individual monitoring due to its high cost, high risk of complications, and whether to perform fine examination is usually based on the subjective judgment of the physician. In recent years, the development of artificial intelligence (AI) has improved the accuracy of medical image diagnosis, and deep learning (DL) models trained on large samples can accurately identify liver cancer from FLL images of gray-scale ultrasound. The sensitivity and specificity of the AI-based liver cancer identification scheme in screening HCC high-risk patients result in insufficient screening accuracy, and the training image sample size required for training the neural network is large, and the cost of obtaining the training samples is high. There is currently no particularly effective AI scheme for screening AFP-negative HCC. SUMMARY

[0004] To address the insufficient sensitivity and specificity of current HCC high-risk population screening programs using grayscale ultrasound images to screen high-risk HCC patients, the present invention provides a fast, convenient, and highly accurate auxiliary screening program for hepatocellular carcinoma based on an artificial intelligence method based on deep learning (DL) modeling.

[0005] A first aspect of the present invention provides an auxiliary screening method for hepatocellular carcinoma. The method comprises: inputting a grayscale ultrasound image of a liver lesion of a subject to be screened into a trained hepatocellular carcinoma (HCC) diagnostic model for screening, thereby obtaining a HCC screening result for the subject; the HCC diagnostic model is trained using training image samples generated by the grayscale ultrasound image of the liver lesion to train a neural network model; the grayscale ultrasound image of the liver lesion at least includes a grayscale ultrasound image of a liver lesion that is negative for serum alpha-fetoprotein (AFP) and pathologically diagnosed as HCC.

[0006] Furthermore, the method also includes: marking all collected liver-occupying grayscale ultrasound images as case group images and control group images according to whether the pathological diagnosis results are for hepatocellular carcinoma, and dividing them into a modeling set and an independent test set according to the purpose of modeling. Among them, the control group is a set of liver-occupying grayscale ultrasound images diagnosed as not having hepatocellular carcinoma; the case group is a set of liver-occupying grayscale ultrasound images diagnosed as having hepatocellular carcinoma; the modeling set is used to generate the training image samples; the independent test set is used to perform performance testing and verification on the hepatocellular diagnosis model, which is composed of case group images and control group images with negative serum alpha-fetoprotein test results. Preferably, the collected liver-occupying grayscale ultrasound images are divided into a modeling set and an independent test set at a ratio of 3:1, and the ratio of the number of case group images to control group images in the modeling set and the independent test set is 1:1 (the relevant ratio is roughly sufficient).

[0007] Furthermore, the method further includes converting all collected grayscale ultrasound images into JPG format and saving them as NumPy multidimensional arrays, and normalizing the NumPy multidimensional arrays. The training samples are obtained by performing an expansion operation on all grayscale ultrasound images in the modeling set as source images, and the images generated by the expansion operation inherit the annotation results of the corresponding source images. The expansion operation includes one or both of random cropping and angular rotation transformation. By performing the above expansion operation on grayscale ultrasound images, the number of grayscale ultrasound images used as training samples can be increased, and the cost of collecting training samples can be reduced.

[0008] Furthermore, the training of the neural network model to obtain the hepatocellular carcinoma diagnostic model includes: using the training samples to perform multiple rounds of training and verification on multiple neural network models respectively; constructing the hepatocellular carcinoma diagnostic model based on one of the neural network models with the highest average accuracy in the multiple rounds of verification and the model parameters corresponding to the highest accuracy of the verification results. Preferably, the multiple neural network models are: retaining the part before the average pooling layer of the neural network framework models Xception, Resnet, Desnet, Mobilenet and Inception, inserting a dropout layer into the multi-layer perceptron that receives the output tensor of the average pooling layer of the above-mentioned multiple neural network framework models, and modifying the number of input channels of the first convolutional layer of the multiple neural network framework models to 1 and setting the number of neurons in the last layer of the neural network to 1 to obtain multiple neural network models.

[0009] Furthermore, the training samples are used to perform multiple rounds of training and verification on various neural network models, which is achieved by: using a five-fold cross-validation method, cyclically dividing the training samples into training sets and verification sets in a ratio of 4:1, using the training sets obtained from each division to train the various neural network models and update the model parameters, using the verification sets obtained from the same division to verify the various trained neural network models to obtain verification results, and then entering the next cycle until all five cycles are completed. Accordingly, by statistically analyzing the accuracy and average accuracy of the verification results of each of the various neural network models during the five cycles, the hepatocellular carcinoma diagnostic model is constructed based on the model with the highest average accuracy among the various neural network models and the model parameters when the verification results have the highest accuracy.

[0010] Furthermore, the training sets divided each time are used to train the multiple neural network models and update the model parameters, including: according to each of the multiple neural network models, the probability value of each image in the training set divided each time belonging to the control group image and the case group image, and the group information manually annotated by the image are calculated using the binary cross entropy loss function to obtain the corresponding difference, using the difference as the parameter of the Adam optimization algorithm to calculate the new parameters of the neural network model, and using the new parameters to update the parameters of the neural network model.

[0011] Corresponding to the above method, the second aspect of the present invention provides a hepatocellular carcinoma auxiliary screening system, which includes a processor, a memory and a grayscale ultrasound image acquisition interface; the grayscale ultrasound image acquisition interface is used to obtain a grayscale ultrasound image of the liver space-occupying object to be screened; the processor executes the computer program in the memory to implement the diagnosis of hepatocellular carcinoma on the acquired grayscale ultrasound image of the liver space-occupying object using the above-mentioned hepatocellular carcinoma auxiliary screening method.

[0012] The technical solution provided by this invention trains multiple neural network models using training samples obtained through an expansion operation, and selects the one with the highest diagnostic accuracy and stability to construct a hepatocellular carcinoma diagnostic model. This hepatocellular carcinoma diagnostic model has high sensitivity and specificity for HCC screening / identification in grayscale ultrasound images of liver masses with negative AFP. It can provide a more reliable reference for clinical HCC diagnosis, making it possible to detect early-stage HCC with negative AFP. It is also suitable for monitoring a large number of individuals. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a schematic diagram of an embodiment of the training process of the HCC diagnostic model used in the auxiliary screening method for hepatocellular carcinoma provided by the present invention. DETAILED DESCRIPTION

[0014] In order to make the technical problems, technical solutions and beneficial effects solved by the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0015] The present invention provides an auxiliary screening method for hepatocellular carcinoma, comprising: inputting a grayscale ultrasound image of a liver space-occupying subject to be screened into a trained hepatocellular carcinoma (HCC) diagnostic model for screening, thereby obtaining a HCC screening result for the subject; the HCC diagnostic model is trained using training image samples generated by the grayscale ultrasound image of the liver space-occupying subject to obtain a neural network model; the grayscale ultrasound image of the liver space-occupying subject at least includes a grayscale ultrasound image of the liver space-occupying subject that is negative for serum alpha-fetoprotein (AFP) and pathologically diagnosed as HCC.

[0016] like Figure 1 As shown, the construction process of the hepatocellular carcinoma diagnostic model includes the following stages in sequence: data preparation stage, data enhancement, construction of the hepatocellular carcinoma diagnostic model stage, independent testing of the hepatocellular carcinoma diagnostic model and deployment stage.

[0017] The data preparation phase includes retrospectively collecting preoperative liver ultrasound grayscale images of lesions from HCC patients and patients with benign liver masses. The images can be in various formats, including JPG, JPEG, or DICOM. The collected grayscale ultrasound images are labeled as case and control images based on whether they are diagnosed with hepatocellular carcinoma by pathology, and divided into a modeling set and an independent test set based on modeling purposes. The images are then uniformly formatted and normalized for subsequent model training and validation.

[0018] The control group is a set of grayscale ultrasound images of liver lesions diagnosed as not suffering from hepatocellular carcinoma; the case group is a set of grayscale ultrasound images of liver lesions diagnosed as suffering from hepatocellular carcinoma. The modeling set is used to generate the training image samples; the independent test set is used to perform performance testing and verification on the hepatocellular diagnosis model, which is composed of case group images and control group images with negative serum alpha-fetoprotein test results. Preferably, the collected grayscale ultrasound images of liver lesions are divided into a modeling set and an independent test set in a ratio of 3:1, and the ratio of the number of case group images to the number of control group images in the modeling set and the independent test set is approximately 1:1.

[0019] The formatting and normalization process involves converting all collected grayscale ultrasound images into JPG format and saving them as NumPy multidimensional arrays, followed by normalization of the NumPy multidimensional arrays. Normalizing the NumPy multidimensional arrays maps pixel values ​​from 0-255 to the range 0-1, eliminating the dimensionality effects between pixels and accelerating the convergence of subsequent model training.

[0020] After the data preparation phase is completed, the data enhancement phase begins. This phase includes: reading all grayscale ultrasound images in the modeling dataset as source images, performing random cropping, performing one or two angular rotation transformations to generate more images, and having the newly generated images inherit the annotation results of the corresponding source images, thereby expanding the number of grayscale ultrasound images in the modeling dataset. All grayscale ultrasound images obtained after expansion are used as training image samples for the neural network model. For example, the read grayscale ultrasound images are randomly cropped and subjected to 30-degree, 60-degree, 90-degree, and 120-degree rotation transformations. Through the processing in the data enhancement phase, the limited grayscale ultrasound image data can produce the equivalent value of a large training sample.

[0021] After the training image samples are obtained in the data enhancement stage, the stage of building a hepatocellular carcinoma diagnosis model is entered. The stage of building a hepatocellular carcinoma diagnosis model includes the following steps:

[0022] S1. Adaptively modifying multiple neural network framework models to obtain the multiple neural network models: Figure 1As shown, in one embodiment, the multiple neural network framework models are Xception, Resnet, Desnet, Mobilenet and Inception. The parts before the average pooling layer of the neural network framework models Xception, Resnet, Desnet, Mobilenet and Inception are retained, and a dropout layer is inserted into the multi-layer perceptron that receives the output tensor of the average pooling layer of the above-mentioned multiple neural network framework models, and the number of input channels of the first convolution layer of the multiple neural network framework models is modified to 1, and the number of neurons in the last layer of the neural network is set to 1 to obtain multiple neural network models.

[0023] S2. Construct a hepatocellular carcinoma diagnostic model based on the training image samples and the multiple neural network models: using a five-fold cross-validation method, the training samples are cyclically divided into a training set and a validation set in a ratio of 4:1. The multiple neural network models are trained and the model parameters are updated using the training set obtained from each split. The trained multiple neural network models are validated using the validation set obtained from the same split to obtain validation results, and the next cycle is entered until all five cycles are completed. The accuracy and average accuracy of the validation results of each of the multiple neural network models on the validation sets obtained from the five cycles are calculated. The hepatocellular carcinoma diagnostic model is constructed based on the model with the highest average accuracy among the multiple neural network models and the model parameters with the highest validation results. For the constructed hepatocellular carcinoma diagnostic model, the grayscale ultrasound image is directly input without any preprocessing. After feature extraction, it is input into the average pooling layer, and then passes through the fully connected layer, the dropout layer, and the fully connected layer with 1 neuron in sequence to output the corresponding HCC diagnostic result.

[0024] Furthermore, the training sets divided each time are used to train the multiple neural network models and update the model parameters, including: according to each of the multiple neural network models, the probability value of each image in the training set divided each time belonging to the control group image and the case group image, and the group information manually annotated by the image are calculated using the binary cross entropy loss function to obtain the corresponding difference, using the difference as the parameter of the Adam optimization algorithm to calculate the new parameters of the neural network model, and using the new parameters to update the parameters of the neural network model.

[0025] After the HCC diagnostic model is built, the independent testing and deployment phase begins. This phase primarily involves verifying the diagnostic performance of the HCC diagnostic model and deploying the model.

[0026] Validating the diagnostic performance of the HCC diagnostic model involves calculating the accuracy, sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve (AUC) of the HCC diagnostic model for both the control and case images in the independent test set, thereby verifying the diagnostic efficacy of the HCC diagnostic model. Finally, the saved HCC intelligent diagnostic model architecture and model parameters are deployed to the appropriate device, for example, directly embedded in a hospital ultrasound device, a standard computer, or a patient's mobile phone.

[0027] The following indicators were used to verify the diagnostic performance of the hepatocellular carcinoma diagnostic model:

[0028] Sensitivity (SEN) = TP / (TP + FN);

[0029] Specificity (SPE) = TN / (TN + FP);

[0030] Accuracy (ACC) = (TP + TN) / (TP + FP + TN + FN);

[0031] Positive predictive value (PPV) = TP / (TP + FP);

[0032] Negative predictive value (NPV) = TN / (TN + FN);

[0033] F1 score = 2*PPV*SEN / (PPV+SEN);

[0034] False positive rate (FPR) = FP / (FP+TN);

[0035] False negative rate (FNR) = FN / (TP = FN);

[0036] Among them, TP (True Positives) is the true positive / true positive, the probability that the hepatocellular carcinoma diagnostic model correctly predicts a positive example as a positive example; FN (False Negatives) is the false negative / false negative, the probability that the hepatocellular carcinoma diagnostic model incorrectly predicts a positive example as a negative example; TN (True Negatives) is the true negative / true negative, the probability that the hepatocellular carcinoma diagnostic model correctly predicts a negative example as a negative example; FP (False Positives): is the false positive / false positive, the probability that the hepatocellular carcinoma diagnostic model incorrectly predicts a negative example as a positive example.

[0037] The following table is a schematic diagram of the performance test results of the hepatocellular carcinoma diagnosis model constructed in an embodiment of the present application. In this embodiment, the Xception model is selected through five-fold cross-validation, and the model parameters of the model with the highest verification accuracy are used to construct the hepatocellular carcinoma diagnosis model. The hepatocellular carcinoma diagnosis model has very high sensitivity, area under the ROC curve, and high specificity for AFP-negative liver occupying gray-scale ultrasound images. The model has a high HCC diagnosis accuracy for AFP-negative liver occupying gray-scale ultrasound images. The test result data corresponding to the above performance test parameters of the model are shown in the following table.

[0038]

[0039] After the performance test reaches the expected result, the obtained hepatocellular carcinoma diagnosis model is deployed to the corresponding diagnostic equipment, computer, terminal, etc. by software deployment to provide the patient or ultrasonographer with HCC auxiliary diagnosis.

[0040] Correspondingly, the present application also provides a hepatocellular carcinoma auxiliary screening system, which comprises a processor, a memory, and a gray-scale ultrasound image acquisition interface; the gray-scale ultrasound image acquisition interface is used to acquire a liver occupying gray-scale ultrasound image of a subject to be screened; the processor realizes hepatocellular carcinoma diagnosis on the acquired liver occupying gray-scale ultrasound image by executing a computer program in the memory using the hepatocellular carcinoma auxiliary screening method described above. The memory includes but is not limited to ROM, RAM, optical disc, NorFlash, etc.

[0041] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions provided by the present application.

Claims

1. A method for auxiliary screening of hepatocellular carcinoma, characterized in that: The method includes: inputting a liver lesion-occupying grayscale ultrasound image of a subject to be screened into a trained hepatocellular carcinoma (HCC) diagnostic model for screening, thereby obtaining a HCC screening result for the subject; the HCC diagnostic model uses the training image samples to perform multiple rounds of training and verification on multiple neural network models, and constructing a neural network model based on the neural network model with the highest average accuracy among the multiple rounds of verification and the model parameters corresponding to the highest accuracy of the verification result; The liver-occupying grayscale ultrasound images are divided into a modeling set and an independent test set according to the modeling purpose; the modeling set is used to generate the training image samples; the independent test set is used to perform performance testing and verification on the liver cell diagnosis model, which is composed of case group images and control group images with negative serum alpha-fetoprotein test results; the training image samples are obtained by performing an expansion operation on all grayscale ultrasound images in the modeling set, and the expansion operation retains one or two of random cropping and angle rotation transformation; the liver-occupying grayscale ultrasound images at least include liver-occupying grayscale ultrasound images with negative serum alpha-fetoprotein (AFP) test results and pathological diagnosis of hepatocellular carcinoma.

2. The method according to claim 1, wherein The method also includes: manually labeling all collected grayscale ultrasound images of liver lesions as case group images or control group images according to whether they are pathologically diagnosed as suffering from hepatocellular carcinoma; converting all collected grayscale ultrasound images into JPG format and saving them as NumPy multidimensional arrays, and normalizing the NumPy multidimensional arrays; wherein the control group is a set of grayscale ultrasound images of liver lesions diagnosed as not suffering from hepatocellular carcinoma; and the case group is a set of grayscale ultrasound images of liver lesions diagnosed as suffering from hepatocellular carcinoma.

3. The method according to claim 2, wherein The ratio of the number of images in the modeling set to the number of images in the independent test set is 3:1, and the ratio of the number of case group images in the independent test set to the number of control group images is 1:

1.

4. The method according to any one of claims 1 to 3, wherein The multiple neural network models are: retaining the part before the average pooling layer of the neural network framework models Xception, Resnet, Desnet, Mobilenet and Inception, inserting a dropout layer into the multilayer perceptron that receives the output tensor of the average pooling layer of the above-mentioned multiple neural network framework models, and modifying the number of input channels of the first convolutional layer of the multiple neural network framework models to 1 and setting the number of neurons in the last layer of the neural network to 1 to obtain multiple neural network models.

5. The method according to claim 4, wherein The multiple neural network models are trained and verified respectively, and the hepatocellular carcinoma diagnosis model is constructed based on the neural network model with the highest average accuracy in the verification results and the model parameters corresponding to the highest accuracy in the verification results, specifically: Using a five-fold cross-validation method, the training image samples are cyclically divided into a training set and a validation set in a ratio of 4:1, the multiple neural network models are trained and the model parameters are updated using the training set obtained each time, and the multiple neural network models after training are respectively validated using the validation set obtained in the same split to obtain validation results; and the next cycle is entered until all five cycles are completed; The accuracy and average accuracy of the verification results of the five cycles of each of the multiple neural network models are counted separately; and the hepatocellular carcinoma diagnosis model is constructed based on the one with the highest average accuracy among the multiple neural network models and the model parameters when the verification result has the highest accuracy.

6. The method according to claim 5, wherein The method uses the training sets divided each time to train the multiple neural network models and update the model parameters, including: according to each of the multiple neural network models, the probability value of each image in the training set divided each time belonging to the control group image and the case group image, and the group information manually annotated by the image are calculated using the binary cross entropy loss function to calculate the corresponding difference, using the difference as the parameter of the Adam optimization algorithm to calculate the new parameters of the neural network model, and using the new parameters to update the parameters of the neural network model.

7. The method according to claim 6, wherein The activation functions of the various neural network models are selected as the nonlinear function sigmoid.

8. A hepatocellular carcinoma auxiliary screening system, characterized in that: The system includes a processor, a memory, and a grayscale ultrasound image acquisition interface; the grayscale ultrasound image acquisition interface is used to acquire a grayscale ultrasound image of a liver space-occupying portion to be screened; the processor executes a computer program in the memory to implement hepatocellular carcinoma diagnosis using the hepatocellular carcinoma auxiliary screening method described in any one of claims 1 to 7 on the acquired grayscale ultrasound image of the liver space-occupying portion.

Images