Early cancer position marking method and device based on spectral CT image

Abstract

The present application relates to the technical field of image processing, and discloses a kind of early cancer position marking method and device based on spectral CT image, to solve the problem of poor accuracy of existing early cancer identification method, scheme mainly includes: obtain multiple medical image samples, respectively on each medical image sample in plain scan period, arterial phase and venous phase corresponding spectral CT image is carried out image registration and fusion;Based on the region segmentation model pre-trained, in each fused spectral CT image, the region image to be marked is segmented, if early cancer lesion exists in the region image to be marked, early cancer position is manually marked in the corresponding region image to be marked;All spectral CT image corresponding region image to be marked is used as the first training sample to train early cancer segmentation model, early cancer segmentation model marks early cancer position when early cancer lesion exists in the spectral CT image of patient to be marked.The present application improves the accuracy of early cancer position marking, especially suitable for early cancer position marking in digestive tract.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an early cancer position marking method and device based on spectral CT images. BACKGROUND

[0002] With the acceleration of life pace and the increase of life pressure, the phenomenon of long-term improper diet and irregular diet is common, the incidence of chronic diseases of the digestive system is gradually increasing, and the canceration rate of chronic diseases is also increasing. In the human digestive tract, esophagus, stomach and colorectum are places where early cancer is prone to occur. Early cancer is a short name for early stage cancer, which refers to the early stage of cancer. Early stage cancer has a lower degree of malignancy, a smaller tumor size, less invasion of surrounding tissues, no lymph node metastasis or distant organ metastasis, and the symptoms of early stage cancer patients are generally not obvious. For example, according to the international cancer staging system (TNM staging), early gastric cancer includes stage 0 and stage I, wherein stage 0: also known as carcinoma in situ or precancerous lesion, in this stage, cancer cells are limited to the surface of the gastric mucosa and have not invaded the submucosal tissue or other parts. Stage I: cancer cells have invaded the submucosal tissue of the stomach, but have not spread to the adjacent lymph nodes or other organs.

[0003] The existing technology mainly includes the following two ways to identify early cancer of the digestive tract:

[0004] The first is to use endoscopic images to identify early stage cancer. Endoscopy is a technology that introduces a lens into the body through a natural orifice or surgical incision for observation. Endoscopy can provide high-resolution, high-contrast images that can directly observe the fine structures and lesions inside the body. By performing endoscopic examination of the digestive tract, doctors can directly observe abnormal tissues or lesions that may be suspected of early stage cancer. Although endoscopy can directly observe lesions, its scope of application is limited by the instrument, which can only cover specific parts, and endoscopic examination usually requires patients to undergo an uncomfortable or slightly painful process, and for some endoscopic examinations of certain parts, local anesthesia or general anesthesia is required. In addition, since early stage cancer usually forms a small lesion in the body, its morphological characteristics may be similar to normal tissues, or there are some small, difficult-to-detect abnormalities, even with high-resolution imaging technology, it may not be possible to accurately detect these subtle abnormalities.

[0005] The second is to use CT (Computed Tomography) images for early cancer identification. CT is an X-ray-based imaging technology that obtains images of body cross-sections by scanning X-rays from multiple angles. CT images can provide detailed cross-sectional images of various parts of the body, and can well show the density and structure of tissues, including internal organs, blood vessels, and bones. Through the CT images of the digestive tract, doctors can determine whether there are abnormal tissues or lesions that may be early cancer based on clinical experience. However, because the growth rate of cancer is relatively slow in the early stage, cancer cells are usually still confined to the mucosal surface or submucosal layer at this stage, and have not spread widely to the surrounding tissues or lymph nodes, and the tumor size is relatively small, and the required oxygen and nutrients can be obtained from the surrounding normal tissues. Therefore, early cancer has relatively less angiogenesis, and the blood vessel richness is usually low, and other abnormal tissues such as blood vessels are difficult to show obvious signals on conventional CT images, making it difficult for conventional CT to accurately identify early cancer. In addition, the medical imaging technology itself may have some noise and artifacts, which may interfere with the interpretation of the image. In the CT image, the overlap of artifacts and tissues may make the detection of lesions more difficult, increasing the likelihood of misdiagnosis.

[0006] In addition, doctors need to rely on their experience and judgment when diagnosing early cancer. There may be differences between different doctors, and subjective factors will affect the diagnosis results. Even the same doctor may make different judgments on the same image at different times or moods, leading to inconsistency of results, making the accuracy of early cancer identification relatively low. SUMMARY

[0007] The present application aims to solve the problem of poor accuracy in existing early cancer identification methods, and proposes an early cancer location marking method and device based on spectral CT images.

[0008] The technical solution adopted by the present application to solve the above technical problems is:

[0009] In a first aspect, an early cancer location marking method based on spectral CT images is proposed, the method comprising:

[0010] Obtain a plurality of medical image samples, each medical image sample including spectral CT images of the patient's chest and abdomen in the plain scan period, arterial period and venous period, the spectral CT images including single energy imaging images, effective atomic number images, water images and iodine images, the single energy imaging images including a plurality of single energy imaging images at different energy levels;

[0011] Perform image registration and fusion on the corresponding spectral CT images of the plain scan period, arterial period and venous period in each medical image sample, respectively, to obtain the fused spectral CT images corresponding to each medical image sample;

[0012] segmenting the to-be-labeled region image in each fused spectral CT image based on the pre-trained region segmentation model, and manually labeling the early cancer position in the corresponding to-be-labeled region image if there is an early cancer lesion in the to-be-labeled region image;

[0013] taking the to-be-labeled region images corresponding to all the spectral CT images as first training samples, training an early cancer segmentation model according to the first training samples, and labeling the early cancer position in the spectral CT image of the to-be-labeled patient based on the early cancer segmentation model when there is an early cancer lesion in the spectral CT image.

[0014] Further, the image registration and fusion of the corresponding spectral CT images of the plain scan phase, the arterial phase and the venous phase in each medical image sample specifically include:

[0015] For each kind of spectral CT image corresponding to each medical image sample, taking the spectral CT image of the arterial phase as a reference image and taking the spectral CT images of the plain scan phase and the venous phase as floating images, performing image registration on the spectral CT images of the plain scan phase and the venous phase respectively, and then performing image fusion on the spectral CT images of the arterial phase, the plain scan phase and the venous phase to obtain the fused spectral CT image corresponding to each medical image sample.

[0016] Further, the training of the early cancer segmentation model according to the first training samples specifically includes:

[0017] After data preprocessing and data enhancement processing of the first training samples, a first training data set is generated, and the early cancer segmentation model is trained according to the first training data set and based on a stochastic gradient descent algorithm.

[0018] Further, the training method of the region segmentation model includes:

[0019] In the fused spectral CT image corresponding to each medical image sample, a to-be-labeled region is manually labeled, all the spectral CT images with manually labeled to-be-labeled regions are taken as second training samples, after data preprocessing and data enhancement processing of the second training samples, a second training data set is generated, and the early cancer segmentation model is trained according to the second training data set and based on a stochastic gradient descent algorithm.

[0020] Further, the data preprocessing at least includes cropping, resampling and standardization, and the data enhancement processing at least includes rotation, scaling, Gaussian noise addition, Gaussian blur and low resolution simulation.

[0021] Further, the single-energy imaging maps include single-energy imaging maps at energy levels of 40 keV, 50 keV, 60 keV, 70 keV, 80 keV, 90 keV, 100 keV, 110 keV, 120 keV, 130 keV, and 140 keV.

[0022] Further, when the early cancer segmentation model exists in the energy spectrum CT image of the patient to be marked, the early cancer lesion position is marked.

[0023] The energy spectrum CT images of the chest and abdomen of the patient to be marked in the plain scan period, the arterial period, and the venous period are obtained, and the corresponding energy spectrum CT images in the plain scan period, the arterial period, and the venous period are subjected to image registration and fusion, and then input into the region segmentation model. The region segmentation model segments the to-be-marked region image of the fused energy spectrum CT image of the patient to be marked. The to-be-marked region image is input into the early cancer segmentation model. When the early cancer segmentation model exists in the energy spectrum CT image of the patient to be marked, the early cancer position is marked.

[0024] Further, the to-be-marked region image is an esophageal image, a stomach image, or a colorectal image.

[0025] In a second aspect, an early cancer position marking device based on an energy spectrum CT image is provided. The device includes:

[0026] An image acquisition unit is configured to acquire a plurality of medical image samples. Each medical image sample includes energy spectrum CT images of a patient's chest and abdomen in a plain scan period, an arterial period, and a venous period. The energy spectrum CT images include single-energy imaging maps, effective atomic number maps, water maps, and iodine maps. The single-energy imaging maps include a plurality of single-energy imaging maps at different energy levels.

[0027] An image processing unit is configured to perform image registration and fusion on the corresponding energy spectrum CT images in the plain scan period, the arterial period, and the venous period of each medical image sample, respectively, to obtain a fused energy spectrum CT image corresponding to each medical image sample.

[0028] A region segmentation unit is configured to segment a to-be-marked region image in each fused energy spectrum CT image based on a pre-trained region segmentation model. If an early cancer lesion exists in the to-be-marked region image, an early cancer position is manually marked in the corresponding to-be-marked region image.

[0029] An early cancer segmentation unit is configured to use all to-be-marked region images corresponding to the energy spectrum CT images as first training samples, train an early cancer segmentation model based on the first training samples, and mark an early cancer position when the early cancer segmentation model exists in the energy spectrum CT image of the patient to be marked.

[0030] In a third aspect, another early cancer position marking device based on spectral CT images is provided, which comprises a processor and a memory storing program instructions, and the processor is configured to execute the early cancer position marking method based on spectral CT images as described in the first aspect when executing the program instructions.

[0031] The early cancer position marking method and device based on spectral CT images has the following advantages. The early cancer segmentation model is trained based on spectral CT images. Compared with the traditional CT scan which can only provide information about the structure and density of the tissue, the spectral CT scan can also provide information about the composition and energy spectrum of different substances. By measuring and analyzing the blood supply and tissue composition of the tumor region, more accurate image information can be provided, thereby improving the accuracy of early cancer marking and identification. The corresponding images of the plain scan phase, the arterial phase and the venous phase are registered and fused, and the early cancer segmentation model is trained based on the fused images. The correlation in time and space and the complementarity in information of multiple images are utilized, so that the fused image can more comprehensively and clearly describe the scene, further improving the accuracy of early cancer marking and identification. The effective atomic number map, the water map, the iodine map and the single-energy imaging map under different energy levels are used for segmentation model training. Different types of spectral CT images have different effective information, and the segmentation model can automatically extract the effective information in the corresponding images, thereby improving the integrity of the effective information and further improving the accuracy of early cancer marking and identification. In addition, since the early cancer lesion is very small and the characteristics are extremely inconspicuous, directly performing lesion segmentation on the spectral CT image will bring a large number of false positives. The stomach, esophagus and colorectal regions are segmented out by the segmentation model, and then only these regions are screened for early cancer, thereby reducing the screening range, reducing the screening difficulty, reducing the false positive rate and further improving the accuracy of early cancer marking and identification. The first training sample used for the region segmentation model training and the second training sample used for the early cancer segmentation model training are preprocessed and augmented, thereby improving the robustness and generalization ability of the region segmentation model and the early cancer segmentation model, reducing the sensitivity of the model to medical images, reducing the sample imbalance ratio and improving the accuracy of the region segmentation model and the early cancer segmentation model identification, thereby further improving the accuracy of early cancer marking and identification. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 A flowchart of the early cancer position marking method based on spectral CT images according to an embodiment of the present application is shown in FIG. 1.

[0033] Figure 2 Another flowchart of the early cancer position marking method based on spectral CT images according to an embodiment of the present application is shown in FIG. 2.

[0034] Figure 3A schematic diagram of a spectrum curve according to an embodiment of the present application;

[0035] Figure 4 A schematic diagram of a single-energy imaging graph at different energy levels according to an embodiment of the present application;

[0036] Figure 5 A schematic diagram of an effective atomic number graph according to an embodiment of the present application;

[0037] Figure 6 A schematic diagram of a water graph according to an embodiment of the present application;

[0038] Figure 7 A schematic diagram of an iodine graph according to an embodiment of the present application;

[0039] Figure 8 A schematic diagram of image registration and fusion according to an embodiment of the present application;

[0040] Figure 9 A schematic diagram of image registration according to an embodiment of the present application;

[0041] Figure 10 A schematic diagram of image fusion according to an embodiment of the present application;

[0042] Figure 11 A network structure schematic diagram of an early cancer segmentation model corresponding to a stomach region according to an embodiment of the present application;

[0043] Figure 12 A structure schematic diagram of an early cancer position marking device based on a spectral CT image according to an embodiment of the present application;

[0044] Figure 13 Another structure schematic diagram of an early cancer position marking device based on a spectral CT image according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application.

[0046] In some of the flowcharts described in the specification and claims of the present application and in the above-described figures, a plurality of operations are included that occur in a particular order, but it should be clearly understood that the operations can be performed in an order other than that in which they appear or in parallel, and the serial numbers of the operations, such as 101, 102, etc., are merely used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these flowcharts can include more or fewer operations, and the operations can be performed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this paper are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do "first" and "second" represent different types.

[0047] The technical solutions of the embodiments of the present application are suitable for application scenarios that need to identify and mark the location of early cancer in the chest and abdomen of a patient using spectral CT images, such as early cancer location identification in esophagus, stomach, colorectal, etc.

[0048] Current early cancer identification methods usually involve doctors judging whether there is abnormal tissue or lesion tissue suspected of early cancer in the CT images of the chest and abdomen of a patient according to clinical experience. Since the growth rate of cancer is relatively slow in the early stage, cancer cells are usually still limited to the mucosal surface or submucosal layer at this stage and have not spread widely to the surrounding tissues or lymph nodes, and the tumor size is relatively small, and the required oxygen and nutrients can be obtained from the surrounding normal tissues. Therefore, early cancer is relatively less in terms of angiogenesis, and the blood vessel richness is usually low, and traditional CT scanning can only provide information about tissue structure and density, and other abnormal tissues such as blood vessels are difficult to show obvious signals on conventional CT images, making it difficult for conventional CT to accurately identify early cancer. And the current early cancer identification method relies on the experience of doctors, and is highly subjective, which also makes the accuracy of early cancer identification poor.

[0049] In order to improve the accuracy of early cancer recognition, the embodiment of the present application proposes an early cancer position marking method and device based on spectral CT images. The main technical scheme includes: obtaining a plurality of medical image samples, each medical image sample including spectral CT images of the patient's chest and abdomen in the plain scan period, the arterial period and the venous period, the spectral CT images including single energy imaging images, effective atomic number images, water images and iodine images, and the single energy imaging images including a plurality of single energy imaging images at different energy levels; performing image registration and fusion on the corresponding spectral CT images in the plain scan period, the arterial period and the venous period of each medical image sample respectively to obtain the corresponding fused spectral CT images of each medical image sample; based on a pre-trained region segmentation model, segmenting the to-be-marked region image in each fused spectral CT image, and if there is an early cancer lesion in the to-be-marked region image, manually marking the early cancer position in the corresponding to-be-marked region image; taking the to-be-marked region images corresponding to all spectral CT images as first training samples, training an early cancer segmentation model according to the first training samples, and marking the early cancer position when there is an early cancer lesion in the spectral CT image of the to-be-marked patient based on the early cancer segmentation model.

[0050] Specifically, the embodiment of the present application obtains a plurality of spectral CT images (Multi-energy / spectral CT) in the plain scan period, the arterial period and the venous period, performs registration and fusion on the corresponding images in the plain scan period, the arterial period and the venous period, simultaneously utilizes the effective atomic number image, the water image, the iodine image and the single energy imaging image at different energy levels for the fused image, segments the to-be-recognized region by the region segmentation model, and then trains the early cancer segmentation model, so as to construct an early cancer segmentation model capable of automatically marking the early cancer position. Since the spectral CT scanning can provide more accurate image information by measuring and analyzing the blood supply and tissue composition of the tumor region, the correlation in time and space and the complementarity in information of the multiple images in the plain scan period, the arterial period and the venous period, and the different effective information of different types of spectral CT images, in addition, since the early cancer lesion is very small and the characteristics are extremely inconspicuous, directly performing lesion segmentation on the CT image will bring a great false positive, the present application segments the to-be-recognized region by the region segmentation model, thereby reducing the screening range, reducing the screening difficulty and reducing the false positive rate. The early cancer segmentation model obtained by the present application can accurately mark the early cancer position in the multi-spectral CT image automatically, thereby improving the accuracy of early cancer recognition.

[0051] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0052] Figure 1 and Figure 2 A flowchart illustrating an early cancer location marker method based on spectral CT images, provided for embodiments of this application, includes the following steps:

[0053] Step 101: Acquire multiple medical image samples;

[0054] Each medical image sample includes spectral CT images of the patient's chest and abdomen during the plain, arterial, and venous phases. The spectral CT images include single-energy imaging maps, effective atomic number maps, water maps, and iodine maps. The single-energy imaging maps include multiple single-energy imaging maps at different energy levels.

[0055] In practical applications, energy imaging, effective atomic number maps, water maps, and iodine maps of the patient's chest and abdomen can be obtained by performing a plain scan before enhancement and a dual-phase (arterial and venous) enhanced scan on the GSI (Gemstone Spectral Imaging) mode of an energy-dispersive CT scanner / spectral CT scanner. Enhanced scanning can be performed by injecting the non-ionic iodine contrast agent dimerol (400 mg / ml) via the median antecubital vein using a high-pressure injector at a flow rate of 4-5 ml / s and an injection dose of 1.5 ml / kg. The GSI scan mode (threshold 100 HU) should be selected, and the scanning parameters should be: tube voltage 40-140 kVp, instantaneous switching within 0.25 ms, automatic tube current modulation, slice spacing of 0.625 mm, rotation speed of 0.5 s / cycle, and pitch of 0.508:1. Venous or arterial phase energy-dispersive scanning should then be initiated.

[0056] Single-energy imaging is a medical image that can be obtained from an object using a monochromatic X-ray source. Spectral CT can acquire single-energy imaging images in the range of 40keV–140keV in a single pass, thus obtaining several single-energy imaging images at different energy levels. According to the imaging principle of spectral CT, each pixel in a medical image corresponds to a different CT value at different energy levels. Figure 3 The energy spectrum curves provided for the embodiments of this application are in Figure 3 In the diagram, the horizontal axis represents energy levels, and the vertical axis represents CT values. The slope, or rate of change, between any two energy levels in the 40 keV-140 keV range can be calculated from the energy spectrum curve. Imaging images of several single-energy atoms within the 40 keV-140 keV range provide a more comprehensive and accurate representation of this rate of change. Please refer to [link / reference]. Figure 4The embodiment of the application takes 10 keV as an interval to acquire single-energy imaging graphs of the chest and abdomen of the patient in the plain scan period, the arterial period and the venous period at energy levels of 40 keV, 50 keV, 60 keV, 70 keV, 80 keV, 90 keV, 100 keV, 110 keV, 120 keV, 130 keV and 140 keV. The 11 single-energy atomic imaging graphs are used for training of the segmentation model, so that the model training result is more accurate.

[0057] The effective atomic number graph is used to calculate the atomic number of a compound (or mixture) in the tissue by using the attenuation of the X-ray in the CT scanning process. If the absorption coefficient of an element is the same as the absorption attenuation coefficient of a compound or mixture, the atomic number of the element is the effective atomic number of the compound or mixture. By calculating the effective atomic number, the detection, identification and separation of substances can be performed. The effective atomic number graph can well reflect the material composition in the tissue. Please refer to Figure 5 The embodiment of the application acquires the effective atomic number graphs of the chest and abdomen of the patient in the plain scan period, the arterial period and the venous period. The effective atomic number graphs are used for training of the segmentation model, so that the model training result is more accurate.

[0058] The X-ray absorption coefficient of any substance can be determined by the absorption coefficients of any two base substances. Therefore, the attenuation of any substance can be converted into the density of two substances that produce the same attenuation, so that the analysis of the composition of the substance and the separation of the substance can be realized. The base substance pairs with different high and low attenuation are usually selected, such as water and iodine, water and calcium, calcium and iodine, etc. Among them, the most commonly used is water and iodine. Please refer to Figure 6 and Figure 7 The base substance pair used in the embodiment of the application is water and iodine, that is, the water graph and the iodine graph of the chest and abdomen of the patient in the plain scan period, the arterial period and the venous period are acquired. The water graph and the iodine graph are used for training of the segmentation model, so that the model training result is more accurate.

[0059] In actual application, all the spectral CT images above can be exported from the workstation as a DICOM sequence and converted into an NII file in the NIFTY format. The spectral CT images contained in each medical image sample are shown in the following table:

[0060]

[0061] Step 102, respectively, image registration and fusion are performed on the corresponding spectral CT images of the plain scan period, the arterial period and the venous period in each medical image sample, to obtain the fused spectral CT images corresponding to each medical image sample;

[0062] In the embodiment of the application, the steps of image registration and fusion specifically include:

[0063] For each kind of spectral CT image corresponding to each medical image sample, taking the spectral CT image of the arterial phase as a reference image, taking the spectral CT images of the plain scan phase and the venous phase as floating images, respectively performing image registration on the spectral CT images of the plain scan phase and the venous phase, and then performing image fusion on the spectral CT images of the arterial phase, the plain scan phase and the venous phase, a fused spectral CT image corresponding to each medical image sample is obtained.

[0064] It can be understood that medical image registration refers to: for a medical image, seeking a spatial transformation (or a series of spatial transformations) so that corresponding points on another medical image are consistent in space. Such consistency refers to the same anatomical point on the human body having the same spatial position (consistent position, consistent angle, consistent size) on two matching images. The result of registration should make all anatomical points, or at least all points with diagnostic significance and points of interest for surgery, on the two images reach matching. Medical image fusion refers to: using a specific algorithm to combine two or more images into a new image. The fusion result can utilize the spatial correlation and information complementarity of two (or more) images, and make the fused image have a more comprehensive and clear description of the scene, thereby being more conducive to the recognition of the human eye and the automatic detection of machines. Please refer to Figure 8 , a new fused image can be obtained by image registration and image fusion on multiple images.

[0065] Specifically, for image registration, given two images I1(x,y) and I2(x,y), where I1(x,y) is a reference image, and the other I2(x,y) is a floating image, the double mapping exchange between the two images in coordinate position and gray level can be represented as:

[0066] I2(x,y)=g(I1(f(x,y)));

[0067] Where I1(x,y) and I2(x,y) represent the gray values at the corresponding positions respectively, g is a one-dimensional gray transform, and f(x,y) is a two-dimensional spatial coordinate transform, and the best spatial or geometric transform parameters are sought.

[0068] Please refer to Figure 9In the embodiment of the present application, for each medical image sample, the spectral CT image of the arterial phase is taken as a reference image, and the spectral CT images of the plain scan phase and the venous phase are taken as floating images respectively, the floating images and the reference images are input into the image registration structure to perform image registration on the spectral CT images of the plain scan phase and the venous phase, in the image registration structure, the input floating images and the reference images are spliced in the dimension of the channel, first pass through an affine registration network for affine registration, then pass through a convolutional neural network to predict a displacement field from the floating image to the fixed image, then obtain a sampling grid according to the displacement field, and use a spatial conversion network to resample the floating image using the sampling grid to obtain the registered image.

[0069] Referring to Figure 10 In the embodiment of the present application, the spectral CT images of the arterial phase, the image-registered spectral CT images of the plain scan phase and the venous phase are fused based on a convolutional neural network. The image fusion process based on the convolutional neural network includes three steps: focus detection, weight map optimization and image fusion. First, a sliding window with a step of 2 and a size of 16x16 is used to traverse the input image, and an initial weight map is obtained after the input image is processed by the CNN. If the size of the input image is HxW, the size of the initial weight map is ([H / 2]-8+1)x([W / 2]-8+1), that is, the size of the initial weight map is smaller than that of the input image. Therefore, the initial weight map is further optimized to keep the same size as the input image and remove noise points to obtain a final fusion decision map. Finally, the input image is weighted and fused using the decision map to obtain a fused image.

[0070] In the embodiment of the present application, for each medical image sample, the same kind of spectral CT images corresponding to the plain scan phase, the arterial phase and the venous phase are respectively registered and fused by using the above method, so as to obtain the fused effective atomic number map, water map, iodine map and monochromatic imaging images under the energy levels of 40kev, 50kev, 60kev, 70kev, 80kev, 90kev, 100kev, 110kev, 120kev, 130kev and 140kev, respectively. By registering and fusing the corresponding images of the plain scan phase, the arterial phase and the venous phase, the correlation in space and time and the complementarity in information of multiple images can be utilized, so that the fused images can more comprehensively and clearly describe the scene.

[0071] Step 103, based on the pre-trained region segmentation model, segmenting out a to-be-labeled region image in each fused spectral CT image, if there is a precancer lesion in the to-be-labeled region image, manually marking the precancer position in the corresponding to-be-labeled region image;

[0072] In the embodiment of the present application, the training method of the region segmentation model comprises:

[0073] In the fused spectral CT image corresponding to each medical image sample, a to-be-labeled region is manually labeled, all the spectral CT images with the manually labeled to-be-labeled regions are taken as second training samples, after data preprocessing and data enhancement processing are performed on the second training samples, a second training data set is generated, and an early cancer segmentation model is trained according to the second training data set and based on a stochastic gradient descent algorithm.

[0074] In actual application, for each medical image sample, a to-be-labeled region can be manually labeled by a digestive department professional in the fused spectral CT image, for example, an esophagus, stomach or colorectal region is outlined in the fused spectral CT image. After the to-be-labeled region is labeled, it is taken as a second training sample, and in order to enrich the training data, the second training sample is further subjected to data preprocessing and data enhancement processing.

[0075] In the embodiment of the present application, the data preprocessing at least includes cropping, resampling and standardization.

[0076] In the data of the spectral CT image, a large part of the data is empty, that is, all the values are -1024, and this part of the data does not contain any information, and therefore can be compressed. First, a minimum circumscribed cuboid is obtained from the region greater than -1024 in the spectral CT image, and then the voxels in the minimum circumscribed cuboid are cropped out for subsequent processing.

[0077] In the spectral CT image, the picture data is anisotropic, that is, the spacing represented by a single voxel is different in each direction. The embodiment of the present application ignores the difference between the voxels, and needs to resample the data obtained after cropping to make the actual physical distance represented by each voxel equal in each axis.

[0078] For single-energy imaging images, first, the CT value distribution of the background part in each spectral CT image is counted respectively to obtain the CT values at 0.5% and 99.5%, and the CT value of each spectral CT image is limited to within these two values: too small CT value is uniformly set to 0.5% CT value, and too large CT value is uniformly set to 99.5% CT value. Then, the mean and variance of each spectral CT image are calculated respectively, and the CT value minus the mean divided by the variance is the standardized result. For other spectral CT images, the mean and variance of each spectral CT image are counted respectively, and the image value minus the mean divided by the variance is the standardized result.

[0079] In the embodiment of the present application, the data enhancement processing at least includes rotation, scaling, Gaussian noise addition, Gaussian blur and low resolution simulation.

[0080] Rotation and scaling refers to: traversing all spectral CT images, while randomly performing rotation and scaling processing, setting to randomly obtain an angle from U(-15, 15) and a scaling rate from U(0.7, 1.4), and the rotation center is the medical image center point.

[0081] Gaussian noise refers to: traversing all spectral CT images, randomly adding Gaussian noise, adding zero-centered Gaussian noise to each independent pixel in the sample, and the variance of the noise is randomly obtained from U(0, 0.1). Gaussian noise refers to a class of noise whose probability density function obeys Gaussian distribution (i.e. normal distribution).

[0082] Gaussian blur is a kind of linear smoothing filter, which is suitable for eliminating Gaussian noise and is widely used in the noise reduction process of medical image processing. Gaussian blur is a process of weighted average of the entire medical image. The value of each pixel point is obtained by weighted average of itself and other pixel values in the neighborhood. In the embodiment of the application, the specific implementation of Gaussian blur is to use a 3*3 discrete window sliding convolution (Gaussian kernel) to scan each pixel in the spectral CT image, and the weighted average gray value of the pixels in the sliding window neighborhood is used to replace the value of the center pixel point of the sliding window. Traverse all spectral CT images and apply Gaussian blur. The width of the Gaussian kernel is independently sampled from U(0.5, 1.5).

[0083] Low resolution simulation refers to traversing all spectral CT images for low resolution simulation, using nearest neighbor interpolation to reduce the size of the medical image, setting the gray value of the transformed pixel to be equal to the gray value of the nearest input pixel, and then using cubic spline interpolation to enlarge the medical image to the original size. Cubic spline interpolation (Cubic Spline Interpolation) is a smooth curve passing through a series of shape points. The process of obtaining the curve function group is obtained by solving the three bending moment equation group.

[0084] Data augmentation can well improve the performance of the model, which mainly manifests in: it can improve the robustness of the model and reduce the sensitivity of the model to medical images. When the training data are all in a relatively ideal state, some special cases such as occlusion, brightness and blur are prone to misidentification. Adding noise and mask to the training data can improve the robustness of the model. Increasing the training data can improve the generalization ability of the model. It can also avoid sample imbalance. In medical disease recognition, it is easy to have an extremely unbalanced positive and negative sample situation. By using some data augmentation methods on the few samples, the sample imbalance ratio is reduced.

[0085] After data preprocessing and data enhancement processing are performed on the second training sample, a second training data set is generated, and the early cancer segmentation model can be trained according to the second training data set and based on a stochastic gradient descent algorithm. Specifically, the data in the second training data set can be divided into a training set and a test set in a ratio of 8:2, the training set is randomly sampled in a shuffled order with a BatchSize of 16, and the sampled data is subjected to data enhancement according to the above method again with a certain probability, and the Patch-Size is 128*128*128.

[0086] In the stochastic gradient descent algorithm (SGD), the impulse is set to 0.99, the loss function is the cross-entropy loss function, the feature map output by the model is subjected to pixel-by-pixel soft-max, and the cross-entropy is combined:

[0087] wherein the soft-max is defined as:

[0088]

[0089] wherein M represents the number of categories, y c is a vector, and the elements have only two values of 0 and 1, and if the category is the same as the category of the sample, 1 is taken, otherwise 0 is taken, and P c represents the probability that the sample belongs to c.

[0090] wherein the calculation formula of w c is:

[0091]

[0092] wherein N represents the total number of pixels, and N c represents the number of pixels whose GT category is c.

[0093] After the model training is completed, the Dice coefficient can be obtained by comparing the images manually labeled in the test set with the images labeled by the model, and the higher the Dice coefficient, the better the model training effect. When the Dice coefficient is greater than a preset value, the model training is completed to obtain the region segmentation model, and the expression of the Dice coefficient s is as follows:

[0094]

[0095] wherein |X∩Y| is the intersection between X and Y, and |X| and |Y| represent the number of elements of X and Y, respectively.

[0096] After the region segmentation model is obtained, the fused spectral CT images are input into the region segmentation model, and the region image to be labeled, such as an esophagus image, a stomach image or a colorectal image, can be obtained. If there is an early cancer lesion in the region image to be labeled, the early cancer position can be manually labeled by a gastroenterology professional.

[0097] In actual application, a gastroenterology professional can read the upper gastrointestinal endoscopic image of a patient, use the 3D Slicer software to delineate the early cancer lesion region on the segmented region image to be marked according to the position and size of the early cancer lesion described in the endoscopic examination report, and save the delineated region. When delineating the early cancer lesion, the delineated region is as close to the mucosa layer as possible, and needs to avoid bleeding, necrosis, calcification and other regions. Since the position of the early cancer reflected on the single-energy imaging image at the 40kv energy level is the clearest, in actual application, only the position of the early cancer lesion on the single-energy imaging image at the 40kv energy level needs to be determined to determine the lesion positions on other types of spectral CT images, and then the early cancer lesion delineation on all spectral CT images is completed.

[0098] Step 104, taking the region image to be marked corresponding to all spectral CT images as the first training sample, training an early cancer segmentation model according to the first training sample, and marking the early cancer position when the early cancer lesion exists in the spectral CT image of the patient to be marked based on the early cancer segmentation model.

[0099] In the embodiment of the present application, the training step of the early cancer segmentation model specifically includes:

[0100] After data preprocessing and data enhancement processing are performed on the first training sample, a first training data set is generated, and the early cancer segmentation model is trained according to the first training data set and based on the stochastic gradient descent algorithm.

[0101] Specifically, the first training sample can be subjected to data preprocessing and data enhancement processing in the same way as the second training sample, and the early cancer segmentation model can be trained in the same way as the region segmentation model. Similarly, the 3D U-Net convolutional neural network is used, and the only difference is that the training samples are different, that is, the early cancer segmentation model is trained by taking the region image to be marked corresponding to all spectral CT images as the first training sample. The network structure of the early cancer segmentation model corresponding to the stomach region is shown in Figure 11 . The related method can refer to the training method of the region segmentation model, and the embodiment of the present application will not be described again.

[0102] After the early cancer segmentation model is trained, the early cancer position recognition and marking of the spectral CT image of the patient to be marked can be performed, which specifically includes the following steps:

[0103] The spectral CT images of the chest and abdomen of the patient to be marked in the plain scan phase, the arterial phase and the venous phase are acquired, the corresponding spectral CT images in the plain scan phase, the arterial phase and the venous phase are respectively subjected to image registration and fusion, and then input into a region segmentation model, the region segmentation model segments the to-be-marked region image of the to-be-marked patient from the fused spectral CT image, and the to-be-marked region image is input into an early cancer segmentation model, the early cancer segmentation model marks the early cancer position when the early cancer lesion exists in the spectral CT image of the to-be-marked patient.

[0104] In the embodiment of the present application, the spectral CT images of the chest and abdomen of the patient to be marked in the plain scan phase, the arterial phase and the venous phase are acquired, and the same image registration and image fusion as in step 102 are performed, that is, the fused spectral CT image of the to-be-marked patient is obtained, and then input into a region segmentation model, the region segmentation model segments the to-be-marked image of the to-be-marked patient from the spectral CT image, and input into an early cancer segmentation model, when the to-be-marked image of the to-be-marked patient has an early cancer lesion, the early cancer segmentation model automatically identifies the early cancer lesion position and marks the early cancer lesion position in each spectral CT image. In practical application, the early cancer segmentation model can output multiple three-dimensional coordinates reflecting the early cancer position (the number of coordinates is determined according to the size of the occupied voxel of the early cancer), and the reference coordinate system of the three-dimensional coordinates is the coordinate of the CT image.

[0105] The early cancer position marking method based on spectral CT images in the embodiment of the present application acquires multiple spectral CT images in the plain scan phase, the arterial phase and the venous phase of three phases, registers and fuses the corresponding images in the plain scan phase, the arterial phase and the venous phase, simultaneously uses effective atomic number map, water map, iodine map and single energy imaging map under different energy levels for the fused image, segments the to-be-identified region by a region segmentation model, and then trains an early cancer segmentation model, so as to construct an early cancer segmentation model capable of automatically marking the early cancer position. Since spectral CT scanning can provide more accurate image information by measuring and analyzing the blood supply and tissue composition of the tumor region, the correlation in time and space and the complementarity in information of multiple images in the plain scan phase, the arterial phase and the venous phase, and the different effective information of different types of spectral CT images, in addition, since the early cancer lesion is very small and the characteristics are extremely inconspicuous, directly performing lesion segmentation on the CT image will bring a large number of false positives, the present application segments the to-be-identified region by a region segmentation model, thereby reducing the screening range, reducing the screening difficulty, and reducing the false positive rate. The early cancer segmentation model obtained by training can accurately mark the early cancer position in multiple spectral CT images, thereby improving the accuracy of early cancer identification.

[0106] Corresponding Figure 1 And Figure 2The early cancer position marking method based on the spectral CT image, and the embodiment of the application further provides an early cancer position marking device based on a spectral CT image, as shown in the figure. Figure 12 The device can include:

[0107] An image acquisition unit is configured to acquire a plurality of medical image samples, each of which includes spectral CT images of a patient's chest and abdomen in a plain scan period, an arterial period and a venous period, the spectral CT images including single-energy imaging images, effective atomic number images, water images and iodine images, and the single-energy imaging images including a plurality of single-energy imaging images at different energy levels.

[0108] An image processing unit is configured to perform image registration and fusion on the corresponding spectral CT images in the plain scan period, the arterial period and the venous period of each medical image sample, respectively, to obtain a corresponding fused spectral CT image of each medical image sample.

[0109] A region segmentation unit is configured to segment a to-be-marked region image from each fused spectral CT image based on a pre-trained region segmentation model, and manually mark the early cancer position in the corresponding to-be-marked region image if an early cancer lesion exists in the to-be-marked region image.

[0110] An early cancer segmentation unit is configured to take the to-be-marked region images corresponding to all the spectral CT images as first training samples, train an early cancer segmentation model according to the first training samples, and mark the early cancer position in the spectral CT image of a to-be-marked patient when an early cancer lesion exists based on the early cancer segmentation model.

[0111] Corresponding Figure 1 And Figure 2 The early cancer position marking method based on the spectral CT image, and the embodiment of the application further provides another early cancer position marking device based on a spectral CT image, as shown in the figure. Figure 13 The device can include:

[0112] A processor and a memory having program instructions stored therein, the processor and the memory being in communication with each other through a communication bus, and the processor being configured to execute the program instructions to perform the early cancer position marking method based on the spectral CT image.

[0113] It can be understood that the early cancer position marking device based on the spectral CT image of the embodiment of the application is a device for implementing the early cancer position marking method based on the spectral CT image of the embodiment, and for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the part of the device.

Claims

1. A method for early cancer position marking based on spectral CT images, characterized by, The method comprises: Obtaining a plurality of medical image samples, each medical image sample comprising spectral CT images of a patient's chest and abdomen in a plain scan period, an arterial period and a venous period, the spectral CT images comprising single-energy imaging images, effective atomic number images, water images and iodine images, and the single-energy imaging images comprising a plurality of single-energy imaging images at different energy levels; Respectively performing image registration and fusion on the corresponding spectral CT images of the plain scan period, the arterial period and the venous period in each medical image sample to obtain a corresponding fused spectral CT image of each medical image sample; The respective performing of image registration and fusion on the corresponding spectral CT images of the plain scan period, the arterial period and the venous period in each medical image sample specifically comprises: For each kind of spectral CT image corresponding to each medical image sample, taking the spectral CT image of the arterial period as a reference image and taking the spectral CT images of the plain scan period and the venous period as floating images, respectively performing image registration on the spectral CT images of the plain scan period and the venous period, and then performing image fusion on the spectral CT images of the arterial period, the plain scan period and the venous period to obtain a corresponding fused spectral CT image of each medical image sample; Based on a pre-trained region segmentation model, segmenting a to-be-labeled region image from each fused spectral CT image, and if there is a precancer lesion in the to-be-labeled region image, manually labeling the precancer position in the corresponding to-be-labeled region image; Taking the to-be-labeled region images corresponding to all the spectral CT images as first training samples, training a precancer segmentation model according to the first training samples, and based on the precancer segmentation model, labeling the precancer position when there is a precancer lesion in the spectral CT image of a to-be-labeled patient.

2. The early cancer position marking method based on spectral CT images according to claim 1, characterized by, The training of the precancer segmentation model according to the first training samples specifically comprises: After data preprocessing and data enhancement processing of the first training samples, a first training data set is generated, and the precancer segmentation model is trained according to the first training data set and based on a stochastic gradient descent algorithm.

3. The early cancer position marking method based on spectral CT images according to claim 1, characterized by, The training method of the region segmentation model comprises: Manually labeling to-be-labeled regions in the fused spectral CT images corresponding to each medical image sample, taking all the spectral CT images with manually labeled to-be-labeled regions as second training samples, after data preprocessing and data enhancement processing of the second training samples, generating a second training data set, and training the precancer segmentation model according to the second training data set and based on a stochastic gradient descent algorithm.

4. The early cancer position marking method based on spectral CT images according to claim 2 or 3, characterized by, The data preprocessing at least comprises cropping, resampling and standardization, and the data enhancement processing at least comprises rotation, scaling, Gaussian noise addition, Gaussian blur and low-resolution simulation.

5. The early cancer position marking method based on spectral CT images according to claim 1, characterized in that, The single-energy imaging images comprise single-energy imaging images at 40keV, 50keV, 60keV, 70keV, 80keV, 90keV, 100keV, 110keV, 120keV, 130keV and 140keV energy levels.

6. The early cancer position marking method based on spectral CT images according to claim 1, characterized by, The labeling of the precancer lesion position in the spectral CT image of the to-be-labeled patient based on the precancer segmentation model specifically comprises: Spectrum CT images of a chest and abdomen of a patient to be marked are acquired in a plain scan phase, an arterial phase and a venous phase, corresponding spectrum CT images in the plain scan phase, the arterial phase and the venous phase are respectively subjected to image registration and fusion, and then input into a region segmentation model, the region segmentation model segments a to-be-marked region image of the spectrum CT image after fusion of the patient to be marked, and the to-be-marked region image is input into an early cancer segmentation model, the early cancer segmentation model marks an early cancer position when an early cancer lesion exists in the spectrum CT image of the patient to be marked.

7. The early cancer position marking method based on spectral CT images according to claim 1, characterized by, The to-be-marked region image is an esophagus image, a stomach image or a colorectum image.

8. An early cancer position marking device based on spectral CT images, characterized by, The device comprises: An image acquisition unit is configured to acquire a plurality of medical image samples, each medical image sample including spectrum CT images of a chest and abdomen of a patient in a plain scan phase, an arterial phase and a venous phase, the spectrum CT images including single-energy imaging images, effective atomic number images, water images and iodine images, and the single-energy imaging images including a plurality of single-energy imaging images at different energy levels; An image processing unit is configured to respectively perform image registration and fusion on corresponding spectrum CT images in the plain scan phase, the arterial phase and the venous phase in each medical image sample, to obtain a spectrum CT image after fusion corresponding to each medical image sample; The respective image registration and fusion on corresponding spectrum CT images in the plain scan phase, the arterial phase and the venous phase in each medical image sample specifically includes: For each kind of spectrum CT image corresponding to each medical image sample, taking the spectrum CT image in the arterial phase as a reference image and taking the spectrum CT images in the plain scan phase and the venous phase as floating images, performing image registration on the spectrum CT images in the plain scan phase and the venous phase respectively, and then performing image fusion on the spectrum CT images in the arterial phase, the plain scan phase and the venous phase to obtain a spectrum CT image after fusion corresponding to each medical image sample; A region segmentation unit is configured to segment a to-be-marked region image in each spectrum CT image after fusion based on a pre-trained region segmentation model, and manually mark an early cancer position in the corresponding to-be-marked region image if an early cancer lesion exists in the to-be-marked region image; An early cancer segmentation unit is configured to take to-be-marked region images corresponding to all spectrum CT images as first training samples, train an early cancer segmentation model according to the first training samples, and mark an early cancer position when an early cancer lesion exists in the spectrum CT image of the patient to be marked based on the early cancer segmentation model.

9. An early cancer position marking device based on spectral CT images, characterized by, The device comprises a processor and a memory storing program instructions, and the processor is configured to execute the program instructions to perform the early cancer position marking method based on the spectrum CT image according to any one of claims 1 to 7.

Images