Liver hydatid disease screening method and system based on plain scan CT and deep learning

Abstract

The application provides a liver hydatid disease screening method and system based on plain CT and deep learning, belongs to the technical field of intelligent medical treatment, and realizes liver hydatid disease identification based on plain CT by combining a liver segmentation model, a liver hydatid lesion identification model and a liver hydatid screening model, reduces the risk of patients of intake of more radiation due to enhanced CT, reduces the risk of patients of allergy due to injection of contrast agent, and has strong clinical applicability; can realize patient-level diagnosis on the basis of slice-level identification, classification and segmentation, and has interpretability; in combination with the real situation of hydatid disease in clinical diagnosis and treatment and the clinical characteristics of hydatid disease, the identification accuracy is greatly improved, and in addition, the population including healthy people can be identified and screened, and the use range is wide.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent medical treatment, and particularly relates to a liver hydatid disease screening method and system based on plain CT and deep learning, an electronic device, and a storage medium. BACKGROUND

[0002] Echinococcosis, also known as hydatid disease, is a disease caused by the infection of the larva (hydatid cyst) of the fine-grained echinococcus and multilocular echinococcus in humans. This disease is prevalent in many livestock areas around the world. Moreover, hydatid disease can cause great physical and mental suffering and heavy economic burden to patients and their families.

[0003] Current diagnosis and treatment of hydatid disease face the difficulties of early screening and uneven distribution of medical resources. For early screening, the conventional B-ultrasound screening and immunological diagnosis screening have the following shortcomings for hydatid disease: 1) the environment in the farming and pastoral areas is harsh, and medical resources are scarce, making it impossible to carry out long-term hydatid disease screening; 2) the dependence on the experience of doctors is high, and in areas with general medical level, the diagnostic results are difficult to guarantee; and untimely diagnosis can lead to aversion to the disease and missed best treatment opportunities.

[0004] In the prior art, the hydatid disease automatic auxiliary screening system based on hydatid image recognition and segmentation realizes the standardization of the quantitative and diagnostic processes, and achieves the effect of saving medical resources while improving the diagnostic level of hydatid disease. However, there are the following disadvantages:

[0005] 1) plain CT and enhanced CT are mixed together for classification, and the enhanced CT not only causes the patient to be exposed to more radiation, but also causes the patient to have an allergic risk due to the need for injection of contrast agent, which has poor clinical applicability;

[0006] 2) only a certain period is extracted from multiple periods of hydatid disease for segmentation and classification, the coverage is narrow, and the healthy human category without hydatid disease cannot be identified; the real situation of hydatid disease in clinical diagnosis and treatment and the clinical characteristics of hydatid disease are ignored, resulting in low accuracy and poor clinical practical prospect.

[0007] Therefore, there is an urgent need for a liver hydatid disease screening method suitable for liver hydatid disease. SUMMARY

[0008] The present application provides a liver hydatid disease screening method, system, electronic device and storage medium based on plain CT and deep learning, to overcome at least one technical problem existing in the prior art.

[0009] To achieve the above-mentioned purpose, the present application provides a liver hydatid disease screening method based on plain CT and deep learning, which comprises:

[0010] segment the acquired CT image of the patient to obtain a segmented CT image of a liver region of the patient;

[0011] According to the segmented CT image of the liver region, a plurality of continuous horizontal cross-sections of the CT image of the liver region are obtained, and the horizontal cross-sections are input into a preset slice-level hydatid lesion recognition model;

[0012] Based on the slice-level hydatid lesion recognition model, feature extraction is performed on the CT image slices to obtain slice feature maps, and then the position of the hydatid lesion, the slice-level segmentation result of the lesion, and the initial classification result of the lesion on the slice are determined according to the slice feature maps;

[0013] According to the anatomical order of the CT image slices, the obtained slice feature maps are sequentially input into a preset patient-level hydatid disease screening model, the key slice feature maps are screened by using the attention mechanism of the patient-level hydatid disease screening model, and the probability level of the hydatid disease type and the attention map of the slice prediction sequence of the patient are obtained; the slice-level segmentation result of the lesion, the probability level of the hydatid disease type, and the attention map of the slice prediction sequence are output.

[0014] Further, preferably, the training method of the slice-level hydatid lesion recognition model comprises:

[0015] Obtain a hydatid screening CT image dataset; wherein the hydatid screening CT images in the hydatid screening CT image dataset are labeled with picture labels, and the picture labels include cystic hydatid disease regions, vesicular hydatid disease regions, liver occupying disease regions, and non-lesion regions;

[0016] Using a pre-trained liver segmentation model, the hydatid screening CT images in the hydatid screening CT image dataset are segmented to obtain segmented liver region CT images;

[0017] According to the segmented liver region CT images, a plurality of continuous horizontal cross-sections of the CT image of the liver region are obtained, and the slice-level hydatid lesion recognition model is trained and verified according to all the horizontal cross-sections in the liver region CT image and the liver volume;

[0018] Based on the slice-level hydatid lesion recognition model, feature extraction is performed on the CT image slices to obtain slice feature maps; and according to the feature maps, a plurality of regions of interest are obtained;

[0019] The plurality of regions of interest are recognized, segmented, and classified; wherein the recognized frame regression outputs the coordinates of the recognized frame, the classification outputs the type of the hydatid lesion, and the segmentation outputs whether each voxel point on the liver region CT image slice belongs to the lesion;

[0020] The training of the liver hydatid lesion identification model is constrained by using a loss function and is back propagated until convergence.

[0021] Further, preferably, the training method of the patient-level liver hydatid screening model comprises,

[0022] The CT image slices of the liver region are input into the trained slice-level liver hydatid lesion identification model to obtain slice feature maps;

[0023] The slice feature maps are sequentially input into the patient-level liver hydatid screening model based on the sequence model and the attention mechanism according to the anatomical order of the CT image slice sequence;

[0024] The cross-entropy loss function is calculated and back propagated until convergence to a preset range with the target label of the CT image slice sequence as supervision.

[0025] Further, preferably, the loss function is realized by the following formula:

[0026] L=L bbox +L cls +L mask

[0027] Wherein, L cls is a classification loss function, L bbox is a regression loss function, and L mask is a binary cross-entropy loss function.

[0028] Further, preferably, the target label includes a cystic hydatid disease label, a vesicular hydatid disease label, a liver space-occupying disease label, and a normal label.

[0029] Further, preferably, the liver segmentation model comprises a convolution path and a deconvolution path; wherein,

[0030] The convolution path comprises a first convolution layer, a second convolution layer, a first pooling layer, a third convolution layer, a second pooling layer, a fourth convolution layer, a third pooling layer, a dropout layer, and a fifth convolution layer arranged in sequence;

[0031] The deconvolution path comprises a first deconvolution layer, a first up-sampling layer, a second deconvolution layer, a second up-sampling layer, a third deconvolution layer, a third up-sampling layer, and a fourth deconvolution layer arranged in sequence.

[0032] Further, preferably, the liver segmentation model further quantifies the liver volume according to the number of voxel points on the CT image of the liver region, and further determines the liver volume.

[0033] The slice-level liver hydatid lesion identification model determines the lesion volume according to the slice-level segmentation result of the lesion.

[0034] According to the lesion volume and the liver volume, a lesion / liver volume ratio is obtained.

[0035] To solve the above problems, the application further provides a liver hydatid disease screening system based on plain CT and deep learning, comprising:

[0036] A liver segmentation unit is configured to segment the acquired plain CT image of the patient for hydatid screening by using a pre-trained liver segmentation model to obtain a CT image of a segmented liver region.

[0037] A patient-level screening result acquisition unit is configured to sequentially input the obtained slice feature maps into a preset patient-level liver hydatid disease screening model according to the anatomical order of the CT image slices, and use the attention mechanism of the patient-level liver hydatid disease screening model to screen key slice feature maps and obtain the probability level of the liver hydatid disease type and the attention map of the slice prediction sequence of the patient.

[0038] To solve the above problems, the application further provides an electronic device, which comprises:

[0039] A memory configured to store at least one instruction; and

[0040] A processor configured to execute the instruction stored in the memory to implement the steps of the above-mentioned liver hydatid disease screening method based on plain CT and deep learning.

[0041] To solve the above problems, the application further provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the above-mentioned liver hydatid disease screening method based on plain CT and deep learning.

[0042] The liver hydatid disease screening method, system, electronic device and storage medium based on plain CT and deep learning have the following beneficial effects:

[0043] 1) It reduces the risk of patients being exposed to more radiation due to enhanced CT scans and lowers the risk of allergic reactions due to the need for contrast agent injections, making it highly clinically applicable;

[0044] 2) It can achieve patient-level diagnosis based on slice-level identification, classification and segmentation, while also being interpretable; it combines the real situation of echinococcosis in clinical diagnosis and treatment and the clinical characteristics of echinococcosis, greatly improving the accuracy of identification. In addition, it can be used to identify and screen populations including healthy people, making it widely applicable.

[0045] 3) This invention demonstrates powerful image information mining capabilities, enabling the extraction of feature information such as liver lesions. By using manually annotated plain CT images of lesions, it employs convolutional neural networks and recurrent neural network models to identify and classify various echinococcosis lesions. This provides significant assistance to the clinical research, diagnosis, treatment, and prognosis of hepatic echinococcosis, greatly helping primary care physicians improve diagnostic accuracy and reduce their workload. It allows patients to receive a clear diagnosis and precise treatment plan earlier, possessing significant social and public welfare value. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a method for screening hepatic echinococcosis based on plain CT scan and deep learning, according to an embodiment of the present invention.

[0048] Figure 2 This is a schematic diagram illustrating the principle of a liver echinococcosis screening method based on plain CT scan and deep learning, provided in an embodiment of the present invention.

[0049] Figure 3 A schematic diagram illustrating the principle of a liver segmentation model for a liver echinococcosis screening method based on plain CT scan and deep learning, provided in an embodiment of the present invention.

[0050] Figure 4 This is a schematic diagram illustrating the principle of a liver echinococcosis lesion identification model based on plain CT scan and deep learning, provided in an embodiment of the present invention.

[0051] Figure 5 This is a schematic diagram of the slice-level segmentation result of a liver echinococcosis screening method based on plain CT and deep learning provided in an embodiment of the present invention;

[0052] Figure 6 A schematic diagram of an attention map of a slice prediction sequence of a liver hydatid disease screening method based on plain CT and deep learning provided by an embodiment of the present application is shown in FIG. 1.

[0053] Figure 7 A module schematic diagram of a liver hydatid disease screening system based on plain CT and deep learning provided by an embodiment of the present application is shown in FIG. 2.

[0054] Figure 8 An internal structure schematic diagram of an electronic device for implementing a liver hydatid disease screening method based on plain CT and deep learning provided by an embodiment of the present application is shown in FIG. 3.

[0055] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0056] It should be understood that the specific embodiments described herein are merely intended to explain the present application and are not intended to limit the present application.

[0057] Figure 1 A flowchart of a liver hydatid disease screening method based on plain CT and deep learning provided by an embodiment of the present application is shown in FIG. 4. The method can be executed by a system, which can be implemented by software and / or hardware.

[0058] In recent years, the application of frontier technologies such as big data and artificial intelligence in the medical field has become a trend, and deep learning has been widely applied in the field of medical auxiliary diagnosis. Applying deep learning to early diagnosis of liver hydatid disease can alleviate the problem of lack of medical resources in remote areas and save the lives of countless patients. Convolutional neural networks have made remarkable breakthroughs in image classification, detection and many other tasks compared with traditional pattern recognition methods.

[0059] The conventional deep learning research on liver tumors is mostly based on a phase in a liver three-phase dynamic, and the rays taken by the patient are much larger than a single CT plain scan, the liver hydatid disease screening method based on the plain scan CT and deep learning provided by the application is mainly suitable for the plain scan CT scene. It is found in the previous study that the growth of hydatid cysts mainly relies on the growth of many germinal cysts, ascogonium and protoscolex in the germinal layer of echinococcus into the cyst, and the lesion does not have blood supply of the host, and the image of the hydatid disease lesion has little difference before and after CT enhancement. Therefore, the application conceives to carry out related research on deep learning based on the CT plain scan image of the hydatid disease patient. Compared with the conventional deep learning research on liver tumors, the related research on deep learning based on the CT plain scan has the following advantages: 1. Non-invasive to the patient. Avoid the risk of allergy caused by injection of contrast agent and the contrast agent itself; 2. The patient takes less rays. The liver hydatid disease screening method, system, storage medium and electronic device based on the plain scan CT and deep learning provided by the application realize the identification of liver hydatid disease based on the plain scan CT, reduce the risk of the patient taking more radiation due to the enhanced CT, reduce the risk of the patient having allergy due to the need of injection of contrast agent, and have strong clinical applicability; can realize patient-level diagnosis on the basis of slice-level identification, classification and segmentation, and has explainability; combined with the real situation of hydatid disease in clinical diagnosis and treatment and the clinical characteristics of hydatid disease, the identification accuracy is greatly improved, in addition, the population including healthy people can be identified and screened, and the use range is wide; provide great help for clinical research, diagnosis, treatment and prognosis of liver hydatid disease, and to a great extent, can help primary doctors improve the diagnosis accuracy and reduce the workload of doctors.

[0060] It should be noted that the "diagnosis" described in the application should be understood as lesion identification and liver hydatid disease type determination in the context, and should not be understood as disease treatment. Specifically, a liver hydatid disease lesion identification method based on deep learning is provided, which can intelligently identify liver hydatid disease lesions, and then the type and probability of liver hydatid disease of the patient, improve the identification accuracy and the detection rate of hydatid disease patients.

[0061] Figure 1 And Figure 2 The liver hydatid disease screening method based on the plain scan CT and deep learning is described in the whole. Among them, Figure 1 The flowchart of the liver hydatid disease screening method based on the plain scan CT and deep learning provided by an embodiment of the application is shown in the figure; Figure 2 The principle diagram of the liver hydatid disease screening method based on the plain scan CT and deep learning provided by an embodiment of the application is shown in the figure.

[0062] As Figure 1As shown, in the present embodiment, the liver hydatid disease screening method based on plain CT and deep learning includes steps S110-S130.

[0063] S110, using a pre-trained liver segmentation model, segmenting the acquired patient's hydatid screening plain CT image to obtain the CT image of the segmented liver region; S120, according to the CT image of the segmented liver region, obtaining a plurality of continuous horizontal profile slices of the CT image of the liver region, inputting the horizontal profile slices into a pre-set slice-level liver hydatid lesion identification model; based on the slice-level liver hydatid lesion identification model, extracting features from the CT image slices to obtain slice feature maps, and then determining the position of the liver hydatid disease lesion, the slice-level segmentation result of the lesion and the initial classification result of the lesion on the slice; S130, according to the anatomical order of the CT image slices, the obtained slice feature maps are sequentially input into a pre-set patient-level liver hydatid disease screening model, and the attention mechanism of the patient-level liver hydatid disease screening model is used to screen key slice feature maps, and the probability level of the liver hydatid disease type of the patient and the attention map of the slice prediction sequence are obtained; outputting the slice-level segmentation result of the lesion and the probability level of the liver hydatid disease type and the attention map of the slice prediction sequence. It should be noted that in the specific implementation process, the anatomical order of the CT image slices is the order from top to bottom.

[0064] Figure 2 The principle of the liver hydatid disease screening method based on plain CT and deep learning of the present application is described in the whole; the liver hydatid disease screening method based on plain CT and deep learning of the present application is to integrate multi-dimensional information and comprehensively give the recommendation of treatment scheme; a model framework for multi-dimensional information evaluation and integration is established. For example Figure 2 As shown, it mainly includes two parts of slice-level liver hydatid lesion identification model and patient-level liver hydatid disease screening model.

[0065] Figure 2 The left side of the slice-level model outputs the slice-level classification result and the lesion segmentation result. Figure 2 The middle part of the slice-level model is the patient-level diagnosis of the whole patient's hydatid screening plain CT image, and the process is that the liver region is segmented through the liver segmentation model, and then the slice-level classification result of each slice is aggregated through the Bi-GRU network as the output of the patient-level liver hydatid disease screening model. Figure 2 The right part of the slice-level model is the result output, including the patient-level diagnosis of the patient, the slice-level lesion segmentation result of each CT slice and the attention map for assisting the diagnosis of the radiologist. Specifically, the attention mechanism of the patient-level liver hydatid disease screening model can find the key slice that is helpful for the diagnosis of the model, and the key slice can be segmented and classified through the slice-level lesion segmentation and classification unit, thereby providing the doctor with the use.

[0066] Specifically, first, a CT image of a patient is obtained for screening of hydatid disease; then, the CT image of the patient for screening of hydatid disease is input into a liver segmentation model after preprocessing to obtain a CT image of a liver region; a plurality of continuous horizontal profile slices corresponding to the CT image of the liver region are input into a slice-level hydatid disease lesion identification model to obtain a position, a category, segmentation information of a lesion, and a feature map obtained through a convolution network; then, according to an anatomical order (from top to bottom) of the CT image slices, the slice feature maps are sequentially input into a preset patient-level hydatid disease screening model to obtain an output result. The output result includes slice-level segmentation results of the lesion, a probability-level hydatid disease type, and an attention map of a slice prediction sequence.

[0067] That is, in view of the defects in the prior art and clinical needs, the purpose of the present application is to provide a screening system for liver hydatid disease lesion identification, classification and segmentation based on plain CT and deep learning, which can mine feature information such as liver space-occupying, use plain CT images manually labeled with hydatid disease lesions, combine CT images of common liver space-occupying diseases such as liver cysts and normal liver CT images, and use a convolutional neural network and a recurrent neural network model to identify, classify and segment hydatid disease lesions (including cystic and vesicular hydatid disease) in the liver. The steps are described below.

[0068] S110, using a pre-trained liver segmentation model, segmenting the obtained CT image of the patient for screening of hydatid disease to obtain a CT image of a segmented liver region. That is, a Computed Tomography (CT) sequence image of a patient is obtained through a medical instrument, and in this embodiment, the organ is the liver and the lesion is liver hydatid disease. Exemplarily, the technical parameters of the CT plain scan image are as follows: a Phillips 256-layer iCT scanner is selected for CT, the scanning range is the diaphragm top to the inferior vena cava, and the scanning parameters are tube voltage: 120KV, current: 300mA, slice thickness: 5mm, and slice interval: 10mm.

[0069] Then, a computer is used to obtain a CT image sequence for screening of hydatid disease, specifically, this can be achieved based on Computer Vision (CV) technology. Computer vision is a science that studies how to make machines "see". More specifically, it refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further process images so that the computer processing becomes more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and tries to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval and other technologies.

[0070] In this embodiment, a sequence of plain CT images for echinococcosis screening generated by medical instruments is acquired using a camera, computer, or PAD.

[0071] In this embodiment, the computer device inputs the medical image data corresponding to the organ into the liver segmentation model to obtain the corresponding segmentation result. The liver segmentation model is obtained by training and validating through a public dataset; however, the specific implementation method of the liver segmentation model is not limited in this embodiment.

[0072] Figure 3 The principle of the liver segmentation model of the hepatic echinococcosis screening method based on plain CT and deep learning provided in one embodiment of the present invention is described, as follows: Figure 3 As shown, the liver segmentation model can be, but is not limited to, a 3D ResUNet model. The network structure includes convolutional and deconvolutional paths. The convolutional path consists of a first convolutional layer, a second convolutional layer, a first pooling layer, a third convolutional layer, a second pooling layer, a fourth convolutional layer, a third pooling layer, a dropout layer, and a fifth convolutional layer, arranged sequentially. The deconvolutional path consists of a first deconvolutional layer, a first upsampling layer, a second deconvolutional layer, a second upsampling layer, a third deconvolutional layer, a third upsampling layer, and a fourth deconvolutional layer, arranged sequentially. In specific implementation, the liver segmentation model is trained and validated based on a public dataset for liver segmentation and liver volume quantification. The public dataset includes several abdominal CT training samples and several test samples. For example, 131 abdominal CT training samples and 30 test samples are used for training and validating the liver segmentation algorithm. Image information is aggregated using convolutional and pooling layers in the convolutional path, and image resolution is restored using deconvolutional and upsampling layers in the deconvolutional path. After segmenting the liver, the liver volume is quantified by calculating the number of voxels within the liver region. In other words, the liver volume is quantified based on the number of voxels on the plain CT image of the liver region used for hydatid cyst screening.

[0073] The liver region was segmented from the CT image set of echinococcosis screening using a liver segmentation model. Based on the liver segmentation results, the lesion identification model was trained and validated. During training and validation, cystic echinococcosis lesions, vesicular echinococcosis lesions, and other liver space-occupying lesions were assigned different target labels using ITK-SNAP software.

[0074] S120, according to the CT image of the segmented liver region, a plurality of continuous horizontal cross-section slices of the CT image of the liver region are obtained, and the horizontal cross-section slices are input into a preset slice-level liver hydatid lesion recognition model; feature extraction is performed on the CT image slices based on the slice-level liver hydatid lesion recognition model to obtain a slice feature map, and then the position of the liver hydatid disease lesion, the slice-level segmentation result of the lesion and the initial classification result of the lesion on the slice are determined according to the slice feature map.

[0075] The training method of the liver hydatid lesion recognition model comprises: S121, obtaining a hydatid screening plain CT image dataset; wherein the hydatid screening plain CT images in the hydatid screening plain CT image dataset are labeled with picture labels, and the picture labels comprise a cystic hydatid disease region, a vesicular hydatid disease region, a liver occupying disease region and a non-lesion region. In the specific implementation process, 600 cases of hydatid disease cases are provided, of which 300 are cystic hydatid, 300 are vesicular hydatid, 200 are common liver occupying diseases such as liver cysts and liver hemangiomas, and 100 are normal liver patients. Each case contains an abdominal plain CT image. The lesion part of each disease is labeled by a doctor, and each pixel is given a different label. For example, the cystic hydatid disease region is 1, the vesicular hydatid disease region is 2, the liver cyst and other common liver occupying diseases are 3, and the non-lesion region is 0. It should be noted here that the above label is for a single picture, that is, a picture label.

[0076] S122, using the pre-trained liver segmentation model, the hydatid screening plain CT images in the hydatid screening plain CT image dataset are segmented to obtain segmented liver region CT images; the liver volume is quantified according to the liver region CT image to determine the liver volume of the patient; S123, according to the liver volume, a plurality of continuous horizontal cross-section slices of the CT image of the liver region are obtained, and the slice-level liver hydatid lesion recognition model is trained and verified according to all the horizontal cross-section slices in the liver region CT image and the liver volume; S124, feature extraction is performed on the liver region CT image slices based on the slice-level liver hydatid lesion recognition model to obtain a slice feature map; and a plurality of regions of interest are obtained according to the feature map; S125, the plurality of regions of interest are recognized, classified and segmented; wherein the recognition frame regression outputs the coordinates of the recognition frame, the classification outputs the type of liver hydatid disease lesion, and the segmentation outputs whether each voxel point on the horizontal cross-section slice of the liver region CT image belongs to the lesion; S126, the training of the liver hydatid lesion recognition model is constrained by using a loss function and is back propagated until convergence.

[0077] The following will be combined Figure 4 The slice-level liver hydatid lesion recognition model is described in detail. Figure 4The principle of the slice-level liver hydatid lesion identification model of the liver hydatid disease screening method based on plain CT and deep learning provided by an embodiment of the present application is described, as shown in Figure 4 The lesion identification model of the present application is based on Mask RCNN, and mainly includes four parts: (1) a convolutional neural network for extracting feature maps; (2) a fully convolutional neural network for performing bounding box regression; (3) a fully convolutional neural network for performing classification; and (4) a fully convolutional neural network for performing segmentation. The specific process is that the plain CT image is first extracted by the convolutional neural network, such as the feature pyramid network of ResNet-101 (including three layers of convolution, pooling and activation), to obtain the feature map; a plurality of ROI (region of interest) is generated on the feature map, i.e. the region of interest. Then, the ROI is aligned to complete the correspondence between the ROI and the original image, i.e. the ROI is aligned using the bilinear interpolation method; the ROI is classified and segmented using two fully connected neural networks for bounding box regression and classification, respectively. Specifically, the task includes three branches, then the loss function is calculated and back propagation is performed until the network converges, thereby training the lesion identification network. Among them, the bounding box regression task outputs the coordinates of the bounding box, and the MSE is calculated with the position of the target label assigned by the third step as the regression loss function L bbox ; the classification task outputs the classification category, including cystic, vesicular hydatid disease and other liver occupying diseases, and the cross entropy loss function L cls is calculated with the real label; the segmentation task outputs whether each point on the image belongs to the lesion, and the binary cross entropy loss function L mask is calculated by comparing with the real label. The loss function used for training the entire model is L = L bbox + L cls + L mask ; each object in the above process is a single picture in the sequence of hydatid screening plain CT images of the patient. That is, the CT pictures are batch input, the loss function is calculated between the output of the liver hydatid lesion identification model and the picture label mentioned in step S131, and the above network model is trained by the back propagation method.

[0078] Figure 5 The lesion segmentation result of the slice-level liver hydatid lesion identification model is exemplarily illustrated, wherein Figure 5 is a schematic diagram of the slice-level segmentation result of the liver hydatid disease screening method based on plain CT and deep learning provided by an embodiment of the present application; as shown in Figure 5As shown, the leftmost is from three two-dimensional horizontal profile slices randomly selected, the second column is the lesion segmentation manually labeled by doctors, the third column is the automatic classification and segmentation result, and the fourth column is the grad-cam (the area most focused on by the model through gradient observation) of the prediction model. (a) row is the CT slice of cystic lesions. (b) row is the CT slice of bubble lesions. (c) row is the CT slice of liver cystic lesions. It should be noted that Guided Grad-CAM gives an image and a target class as input, propagates the image through the CNN part of the model, and then obtains the original score of the class through the calculation of the specific task. For all classes, except for the gradient of the required class, which is set to 1, the rest is set to zero. Then this signal is backpropagated to the attentional convolutional feature map of interest, where the coarse Grad-CAM localization (blue-red heat map) is calculated, which indicates where the model needs to look to make an accurate decision. Finally, the heat map is multiplied point by point with the guided backpropagation.

[0079] S130, according to the anatomical order of the CT image slices, the obtained slice feature maps are sequentially input into the preset patient-level liver hydatid disease screening model, the key slice feature maps are screened by using the attention mechanism of the patient-level liver hydatid disease screening model, and the probability level of the liver hydatid disease type of the patient and the attention map of the slice prediction sequence are obtained; the slice-level segmentation result of the lesion and the probability level of the liver hydatid disease type and the attention map of the slice prediction sequence are output.

[0080] In one specific embodiment, further comprising, the liver segmentation model quantifies the liver volume according to the number of voxel points on the CT image of the liver region, and then determines the liver volume; the slice-level liver hydatid lesion identification model determines the lesion volume according to the slice-level segmentation result of the lesion; and the lesion / liver volume ratio is obtained according to the lesion volume and the liver volume. In practice, the liver hydatid disease screening method based on plain CT and deep learning of the present application can also provide the lesion / liver volume ratio for the doctor to refer to. In the diagnosis process, the lesion / liver volume ratio can be used to evaluate the severity of the patient's disease.

[0081] In one specific embodiment, after obtaining the trained slice-level liver hydatid lesion identification model, the patient-level liver hydatid disease screening model is further obtained, and the specific training method includes: S131, inputting the liver region CT image slice into the trained slice-level liver hydatid lesion identification model to obtain the slice feature map; S132, sequentially inputting the slice feature map into the patient-level liver hydatid disease screening model based on the sequence model and the attention mechanism according to the anatomical order of the CT image slice sequence; S133, taking the target label of the CT image slice sequence as supervision, calculating the cross-entropy loss function and performing backpropagation until converging to a preset range.

[0082] Specifically, the original feature map is input into the patient-level liver hydatid disease screening model to obtain a fusion feature map, each feature in the fusion feature map is fused with a sequence, and then an attention map is obtained. It should be noted that the sequence model can be a long short-term memory network or a Bi-GRU. The slice-level classification results of each horizontal profile slice are aggregated by the Bi-GRU network. The Bi-GRU network (Bi Gate Recurrent Unit) can model sequence features.

[0083] The target label includes a cystic hydatid disease label, an alveolar hydatid disease label, a liver space-occupying disease label and a normal label. It should be noted that for one patient, the disease type is also manually labeled by a doctor, that is, the disease type suffered by the patient is labeled, such as 1 for suffering from cystic hydatid, 2 for suffering from alveolar hydatid, 3 for suffering from other liver space-occupying diseases, and 0 for not suffering from liver diseases. This label is the label of the patient, that is, the target label can also be called the disease label.

[0084] Figure 6 The attention map of the slice prediction sequence is described. Among them Figure 6 The attention map of the slice prediction sequence of the liver hydatid disease screening method based on plain CT and deep learning provided by an embodiment of the present application is shown. As shown in Figure 6As shown, the slice-level prediction of one cystic patient forms the patient-level prediction, and the result of each slice and the attention score. In the specific implementation process, the four colors in the predicted label sequence represent the four categories in the slice-level prediction, including cystic (pink), cystic (orange), liver cyst (purple) and normal (green). It can be seen that most of the slices of the liver region are normal, because the lesion is only part of the liver. The orange slice in this example has the most number in the lesion, while the pink and purple slices are false positive and false negative predictions. It should be noted that false positives and false negatives are inevitable regardless of the model. The slice attention chart shows that the model automatically learns to give higher weights to true positive slices, thereby reducing the impact of false positive and false negative slices on patient prediction. In the above manner, a trained liver segmentation model, a slice-level liver hydatid lesion identification model and a patient-level liver hydatid disease screening model are finally obtained. Through the liver segmentation model, the liver hydatid lesion identification model and the liver hydatid disease screening model, liver hydatid disease identification based on plain CT is realized, the risk of patients from taking more radiation due to enhanced CT is reduced, the risk of patients from needing to inject contrast medium due to allergy is reduced, and the clinical applicability is stronger. It can realize patient-level diagnosis on the basis of slice-level identification, classification and segmentation, and has explainability. Combined with the real situation of hydatid disease in clinical diagnosis and treatment and the clinical characteristics of hydatid disease, the identification accuracy is greatly improved, in addition, it can identify and screen the population including healthy people, and has a wide range of use. It provides a great help for the clinical research, diagnosis, treatment and prognosis of liver hydatid disease, and to a great extent, it can help primary doctors improve the diagnosis accuracy and reduce the workload of doctors.

[0085] As shown in Figure 7 The present application provides a liver hydatid disease screening system 700 based on plain CT and deep learning, which can be installed in an electronic device. According to the functions implemented, the liver hydatid disease screening system 700 based on plain CT and deep learning can include a liver segmentation unit 710, a slice-level feature extraction unit 720 and a patient-level screening result acquisition unit 730. The units of the present application can also be referred to as modules, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, and are stored in the memory of the electronic device.

[0086] In this embodiment, the functions of each module / unit are as follows:

[0087] The liver segmentation unit 710 is configured to segment the acquired patient's hydatid screening plain CT image using a pre-trained liver segmentation model, and obtain a CT image of the segmented liver region.

[0088] The slice-level feature extraction unit 720 is configured to acquire a plurality of continuous horizontal profile slices of the CT image of the liver region according to the CT image of the segmented liver region, input the horizontal profile slices into a preset slice-level liver hydatid lesion identification model, perform feature extraction on the CT image slices based on the slice-level liver hydatid lesion identification model to obtain slice feature maps, and then determine the position of the liver hydatid disease lesion, the slice-level segmentation result of the lesion, and the initial classification result of the lesion on the slice based on the slice feature maps.

[0089] The patient-level screening result acquisition unit 730 is configured to sequentially input the obtained slice feature maps into a preset patient-level liver hydatid disease screening model according to the anatomical order of the CT image slices, screen key slice feature maps by using the attention mechanism of the patient-level liver hydatid disease screening model, and acquire the probability level liver hydatid disease type of the patient and the attention map of the slice prediction sequence.

[0090] The liver hydatid disease screening system 700 based on plain CT and deep learning provided by the application can realize liver hydatid disease identification based on plain CT by combining the liver segmentation model, the liver hydatid lesion identification model, and the liver hydatid disease screening model, reduce the risk of the patient caused by intake of more radiation due to enhanced CT, reduce the risk of the patient caused by injection of contrast agent, and has strong clinical applicability; can realize patient-level diagnosis on the basis of slice-level identification, classification, and segmentation, and has explainability; combines the real situation of hydatid disease in clinical diagnosis and treatment and the clinical characteristics of hydatid disease, greatly improves the identification accuracy, can identify and screen the crowd including healthy people, has a wide range of use, provides great help for clinical research, diagnosis, treatment, and prognosis judgment of liver hydatid disease, and can greatly help primary doctors to improve the diagnosis accuracy and reduce the workload of doctors.

[0091] As shown in Figure 8 The application provides an electronic device 8 for a liver hydatid disease screening method based on plain CT and deep learning.

[0092] The electronic device 8 can include a processor 80, a memory 81, and a bus, and can further include a computer program stored in the memory 81 and executable on the processor 80, such as a liver hydatid disease screening program based on plain CT and deep learning 82. The memory 81 can include both an internal storage unit of the liver hydatid disease screening system based on plain CT and deep learning and an external storage device. The memory 81 can be used not only to store installed application software and various data, such as the code of the liver hydatid disease screening program based on plain CT and deep learning, but also to temporarily store data that has been output or will be output.

[0093] The memory 81 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 81 can be an internal storage unit of the electronic device 8, such as a mobile hard disk of the electronic device 8. In other embodiments, the memory 81 can also be an external storage device of the electronic device 8, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 81 can include both an internal storage unit and an external storage device of the electronic device 8. The memory 81 can be used to store application software and various data installed in the electronic device 8, such as the code of the liver hydatid disease screening program based on a CT scan and deep learning, and can also be used to temporarily store data that has been output or will be output.

[0094] The processor 80 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of a central processing unit (CPU), a microprocessor, a digital processing chip, a graphics processor, and various control chips, etc. The processor 80 is the control unit of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, executes programs or modules stored in the memory 81 (such as the liver hydatid disease screening program based on a CT scan and deep learning, etc.), and calls data stored in the memory 81, to perform various functions and process data of the electronic device 8.

[0095] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize the connection and communication between the memory 81 and at least one processor 80, etc.

[0096] Figure 8 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 8The illustrated structure does not constitute a limitation on the electronic device 8, which can include fewer or more components than shown, or combine certain components, or have a different arrangement of components.

[0097] For example, although not shown, the electronic device 8 can also include a power source (such as a battery) to power the various components, and preferably the power source can be logically connected to the at least one processor 80 through a power management system, so that the power management system can implement functions such as charge management, discharge management, and power consumption management. The power source can also include one or more DC or AC power sources, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and any other components. The electronic device 8 can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not described here.

[0098] Further, the electronic device 8 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, and the like), and is typically used to establish a communication connection between the electronic device 8 and other electronic devices.

[0099] Optionally, the electronic device 8 can also include a user interface, which can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch, and the like. The display can also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device 8 and to display a visualized user interface.

[0100] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by this structure.

[0101] The memory 81 in the electronic device 8 stores a liver hydatid disease screening program 82 based on plain CT and deep learning, which is a combination of a plurality of instructions and can realize, when running in the processor 80, the following: S110, using a pre-trained liver segmentation model, segmenting the obtained patient's hydatid screening plain CT image to obtain a segmented liver region CT image; S120, according to the segmented liver region CT image, obtaining a plurality of continuous horizontal profile slices of the liver region CT image, inputting the horizontal profile slices into a pre-set slice-level liver hydatid lesion identification model; based on the slice-level liver hydatid lesion identification model, extracting features of the CT image slices to obtain slice feature maps, and then determining the position of the liver hydatid disease lesion, the slice-level segmentation result of the lesion and the initial classification result of the lesion on the slice; S130, according to the anatomical order of the CT image slices, inputting the obtained slice feature maps into a pre-set patient-level liver hydatid disease screening model in sequence, using the attention mechanism of the patient-level liver hydatid disease screening model to screen key slice feature maps, and obtaining the probability level of the liver hydatid disease type of the patient and the attention map of the slice prediction sequence; outputting the slice-level segmentation result of the lesion and the probability level of the liver hydatid disease type and the attention map of the slice prediction sequence.

[0102] Specifically, the specific implementation method of the processor 80 to the above instructions can refer to Figure 1 The description of related steps in the corresponding embodiments will not be repeated here. It needs to be emphasized that, in order to further ensure the privacy and security of the above liver hydatid disease screening program based on plain CT and deep learning, the database high-availability processing data is stored in the nodes of the blockchain where the server cluster is located.

[0103] Further, the modules / units integrated by the electronic device 8, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. The computer readable medium can include any entity or system capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory).

[0104] The embodiment of the present application also provides a computer readable storage medium, which can be nonvolatile or volatile, and stores a computer program, which is executed by a processor to realize the following steps: S110, using a pre-trained liver segmentation model to segment the obtained CT image of a patient in a hydatid screening plain scan to obtain a CT image of a segmented liver region; S120, obtaining a plurality of continuous horizontal profile slices of the CT image of the liver region according to the CT image of the segmented liver region, inputting the horizontal profile slices into a preset slice-level hydatid disease lesion identification model, extracting features of the CT image slices based on the slice-level hydatid disease lesion identification model to obtain slice feature maps, and then determining a position of a hydatid disease lesion, a slice-level segmentation result of the lesion and an initial classification result of the lesion on the slice; S130, sequentially inputting the obtained slice feature maps into a preset patient-level hydatid disease screening model according to an anatomical order of the CT image slices, screening key slice feature maps by using an attention mechanism of the patient-level hydatid disease screening model, and obtaining a probability-level hydatid disease type of the patient and an attention map of a slice prediction sequence; and outputting the slice-level segmentation result of the lesion, the probability-level hydatid disease type and the attention map of the slice prediction sequence.

[0105] Specifically, the computer program is executed by the processor to specifically realize the method, which can refer to the description of the related steps in the liver hydatid disease screening method based on a plain scan CT and deep learning, and details are not described herein.

[0106] In several embodiments provided in the present application, it should be understood that the disclosed devices, systems and methods can be implemented in other manners. For example, the above-described system embodiments are merely illustrative, and the division of the modules is merely a logical function division, and there can be another division manner in actual implementation.

[0107] The modules illustrated as separated components can or can not be physically separated, and the components illustrated as modules can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0108] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0109] It will be obvious to a person skilled in the art that the application is not limited to the details of the above-described exemplary embodiments but can be implemented in other embodiments without departing from the scope of the application.

[0110] The embodiments should therefore be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the description given above, which is therefore intended merely as explanatory and not as restrictive, so that all changes coming within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims should not be construed as limiting the scope of the claims.

[0111] The blockchain referred to in the present application is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. The blockchain is essentially a decentralized database, and is a chain of data blocks associated using cryptographic methods. Each data block contains information about a batch of network transactions, and is used to verify the validity of the information (anti-fake) and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer, etc.

[0112] Furthermore, it is clear that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude the plural. A plurality of units, or systems can be implemented by one unit or system performing the functions of the plurality of units or systems by software or hardware. The terms "first", "second" and the like do not denote any order, quantity, or importance, but are used to identify different components.

[0113] Finally, it should be noted that the above embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the present application. Even though the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalent replaced without departing from the spirit and scope of the present application.

Claims

1. A liver hydatid disease screening method based on plain CT and deep learning, characterized in that, The method comprises the following steps: segmenting the acquired CT image of the patient for hydatid screening using a pre-trained liver segmentation model to obtain a segmented CT image of the liver region; According to the segmented CT image of the liver region, a plurality of continuous horizontal cross-sectional slices of the CT image of the liver region are obtained, and the horizontal cross-sectional slices are input into a pre-set slice-level liver hydatid lesion identification model; Based on the slice-level liver hydatid lesion identification model, the CT image slices are subjected to feature extraction to obtain slice feature maps, and then the position of the liver hydatid disease lesion, the slice-level segmentation result of the lesion, and the initial classification result of the lesion on the slice are determined according to the slice feature maps; According to the anatomical order of the CT image slices, the obtained slice feature maps are sequentially input into a pre-set patient-level liver hydatid disease screening model, the key slice feature maps are screened using the sequence model and attention mechanism of the patient-level liver hydatid disease screening model, and the attention map of the slice prediction sequence and the probability level of the liver hydatid disease type of the patient are obtained based on the attention score; The slice-level segmentation result of the lesion, the probability level of the liver hydatid disease type, and the attention map of the slice prediction sequence are output.

2. The plain CT and deep learning-based hepatic hydatid disease screening method of claim 1, characterized in that, The training method of the slice-level liver hydatid lesion identification model comprises: obtaining a hydatid screening CT image dataset; wherein the hydatid screening CT images in the hydatid screening CT image dataset are labeled with picture labels, and the picture labels include cystic hydatid disease regions, vesicular hydatid disease regions, liver occupying disease regions, and non-lesion regions; segmenting the hydatid screening CT images in the hydatid screening CT image dataset using a pre-trained liver segmentation model to obtain segmented liver region CT images; quantifying the liver volume according to the liver region CT images to determine the liver volume of the patient; According to the liver segmentation result, a plurality of continuous horizontal cross-sectional slices of the CT image of the liver region are obtained, and the slice-level liver hydatid lesion identification model is trained and verified according to all the horizontal cross-sectional slices in the liver region CT image and the liver volume; Based on the slice-level liver hydatid lesion identification model, the liver region CT image slices are subjected to feature extraction to obtain slice feature maps; and a plurality of regions of interest are obtained according to the feature maps; The plurality of regions of interest are subjected to recognition box regression, classification, and segmentation; wherein the recognition box regression outputs the coordinates of the recognition box, the classification outputs the type of liver hydatid disease lesion, and the segmentation outputs whether each voxel point on the liver region CT image slice belongs to the lesion; The training of the liver hydatid lesion identification model is constrained using a loss function and is subjected to back propagation until convergence.

3. The plain CT and deep learning-based hepatic hydatidosis screening method of claim 2, wherein, The training method of the patient-level liver hydatid disease screening model comprises, inputting the liver region CT image slices into the trained slice-level liver hydatid lesion identification model to obtain slice feature maps; inputting the slice feature maps into the patient-level liver hydatid disease screening model based on the sequence model and the attention mechanism in the anatomical order of the CT image slice sequence; The cross-entropy loss function is calculated and back propagation is performed until convergence to a preset range with the target label of the CT image slice sequence as supervision. 4.The liver hydatid disease screening method based on plain CT and deep learning of claim 2, characterized in that, The loss function is realized by the following formula: L = L bbox +L cls +L mask wherein, L cls is a classification loss function, L bbox is a regression loss function, L mask is a binary cross-entropy loss function.

5. The liver hydatid disease screening method based on plain CT and deep learning according to claim 3, characterized in that, The target label comprises a cystic hydatid disease label, a vesicular hydatid disease label, a liver space-occupying disease label and a normal label. 6.The liver hydatid disease screening method based on plain CT and deep learning of claim 1, wherein, The liver segmentation model comprises a convolution path and a deconvolution path; wherein, The convolution path comprises a first convolution layer, a second convolution layer, a first pooling layer, a third convolution layer, a second pooling layer, a fourth convolution layer, a third pooling layer, a dropout layer and a fifth convolution layer arranged in sequence; The deconvolution path comprises a first deconvolution layer, a first up-sampling layer, a second deconvolution layer, a second up-sampling layer, a third deconvolution layer, a third up-sampling layer and a fourth deconvolution layer arranged in sequence.

7. The plain CT and deep learning-based hepatic hydatidosis screening method of claim 1, wherein, Further comprising, The liver segmentation model quantifies the liver volume according to the number of voxel points on the CT image of the liver region, and further determines the liver volume; The slice-level liver hydatid lesion recognition model determines the lesion volume according to the slice-level segmentation result of the lesion; According to the lesion volume and the liver volume, the lesion / liver volume ratio is obtained. 8.A liver hydatidosis screening system based on plain CT and deep learning, characterized in that, It comprises: A liver segmentation unit is configured to segment the obtained plain CT image of the patient for hydatid screening by using a pre-trained liver segmentation model to obtain a CT image of a segmented liver region; a slice-level feature extraction unit is configured to obtain a plurality of continuous horizontal cross-sectional slices of the CT image of the liver region according to the CT image of the segmented liver region, input the horizontal cross-sectional slices into a preset slice-level liver hydatid lesion recognition model, extract slice features from the CT image slices based on the slice-level liver hydatid lesion recognition model to obtain a slice feature map, and further determine the position of the liver hydatid disease lesion, the slice-level segmentation result of the lesion and the initial classification result of the lesion on the slice according to the slice feature map; A patient-level screening result acquisition unit is configured to input the obtained slice feature map into a preset patient-level liver hydatid disease screening model in sequence according to the anatomical order of the CT image slices, screen key slice feature maps by using a sequence model and an attention mechanism of the patient-level liver hydatid disease screening model, and obtain the probability level of the liver hydatid disease type and the attention map of the slice prediction sequence based on the attention score; The slice-level segmentation result of the lesion, the probability level of the liver hydatid disease type and the attention map of the slice prediction sequence are outputted.

9. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected in communication with the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps in the liver hydatid disease screening method based on plain CT and deep learning according to any one of claims 1 to 7.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, implements the liver hydatid disease screening method based on plain CT and deep learning as claimed in any one of claims 1 to 7.

Images