Lesion detection method and device based on CT image and computer readable storage medium

Abstract

A lesion detection method and device based on CT images and a computer readable storage medium, wherein the method comprises: processing a chest CT image to obtain a lung image, performing lesion segmentation on the lung image to obtain a segmentation result corresponding to each lesion region in the lung image, and synchronously obtaining edge point position information of a predicted lesion region contained in multiple two-dimensional slices of the lung image, obtaining corresponding three-dimensional edge point position information according to the edge point position information, performing three-dimensional reconstruction on the predicted lesion region according to the three-dimensional edge point position information, obtaining multiple three-dimensional predicted lesion regions, and determining a detection result of each lesion region in the lung image according to the three-dimensional predicted lesion regions, and fusing the segmentation result and the detection result to obtain the position and size of each lesion region in the chest CT image. The method can improve the speed and accuracy of lesion detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus and computer-readable storage medium for lesion detection based on CT images. Background Technology

[0002] Detecting the number, location, and size of pulmonary nodules based on chest CT (Computed Tomography) images is an important means of inferring the characteristics of pulmonary lesions.

[0003] In existing technologies, deep learning algorithms or segmentation models are typically used to detect nodule locations first, and then segmentation is performed. However, this detection method usually has low accuracy. Summary of the Invention

[0004] This application provides a lesion detection method, apparatus, electronic device, and computer-readable storage medium based on CT images, which can simultaneously detect and segment lung lesions to obtain the precise location of the lesions and segmented images containing the lesions, thus solving the problem of low accuracy in lesion detection.

[0005] One embodiment of this application provides a lesion detection method based on CT images, including:

[0006] Chest CT images are processed to obtain lung images;

[0007] The lung image is segmented to obtain segmentation results corresponding to each lesion region within the lung image, and lesion detection is performed on the lung image simultaneously to obtain detection results for each lesion region;

[0008] The segmentation results and the detection results are fused to obtain the location and size of each lesion region in the chest CT image;

[0009] The step of detecting lesions in the lung images to obtain detection results for each lesion region includes:

[0010] The lung image is processed into multiple two-dimensional slices, and lesions are detected in the multiple two-dimensional slices using a preset two-dimensional detection model to obtain the edge point location information of the predicted lesion area contained in each two-dimensional slice.

[0011] Based on the edge point location information of the predicted lesion area, the three-dimensional edge point location information corresponding to the edge point location information is obtained according to the preset three-dimensional reconstruction algorithm, and the predicted lesion area is reconstructed in three dimensions according to the three-dimensional edge point location information to obtain multiple three-dimensional predicted lesion areas.

[0012] Based on each three-dimensional predicted lesion region, the detection results of each lesion region within the lung image are determined.

[0013] One aspect of this application also provides a lesion detection device based on CT images, comprising:

[0014] The processing module is used to process chest CT images to obtain lung images;

[0015] The segmentation module is used to segment the lesions on the lung image to obtain segmentation results corresponding to each lesion region within the lung image;

[0016] The detection module is used to simultaneously detect lesions based on the lung images and obtain the detection results for each lesion region.

[0017] The fusion module is used to fuse the segmentation results and the detection results to obtain the position and size of each lesion region in the chest CT image;

[0018] The detection module is further configured to process the lung image into multiple two-dimensional slices, and perform lesion detection on the multiple two-dimensional slices using a preset two-dimensional detection model to obtain the edge point location information of the predicted lesion region contained in each two-dimensional slice. Based on the edge point location information of the predicted lesion region, a preset three-dimensional reconstruction algorithm is used to obtain the three-dimensional edge point location information corresponding to the edge point location information. The predicted lesion region is then reconstructed in three dimensions according to the three-dimensional edge point location information to obtain multiple three-dimensional predicted lesion regions. Finally, based on each three-dimensional predicted lesion region, the detection result of each lesion region in the lung image is determined.

[0019] One aspect of this application also provides an electronic device, including:

[0020] Memory and processor;

[0021] The memory stores executable computer programs;

[0022] The processor coupled to the memory invokes the executable computer program stored in the memory to perform the steps of the lesion detection method based on CT images described above.

[0023] One aspect of this application also provides a computer-readable storage medium storing a computer program thereon, which, when run by a processor, implements the lesion detection method based on CT images provided in the above embodiments.

[0024] As can be seen from the above embodiments of this application, the present invention, on the one hand, performs lesion segmentation on the lung image obtained by processing a chest CT image, and obtains segmentation results corresponding to each lesion region within the lung image, and simultaneously performs lesion detection on the lung image to obtain detection results for each lesion region, which can improve the speed of lesion detection. On the other hand, by fusing the lesion segmentation results and detection results, the position and size of each lesion region on the chest CT image are obtained. Since the final output position and size of the lesion region are obtained by fusing the synchronously acquired lesion segmentation results and lesion detection results, the two processing results are complementary, thus improving the accuracy and efficiency of lesion detection. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic flowchart of a lesion detection method based on CT images provided in an embodiment of this application;

[0027] Figure 2 A schematic flowchart of a lesion detection method based on CT images provided in another embodiment of this application;

[0028] Figure 3 for Figure 1 A schematic diagram of an implementation flow of step S101 in another embodiment;

[0029] Figure 4 for Figure 3 A schematic diagram of an implementation flow of step S301 in another embodiment;

[0030] Figure 5 A schematic diagram of the bounding box including the left lung region and the right lung region in the lesion detection method based on CT images provided in the embodiments of this application;

[0031] Figure 6 for Figure 3 A schematic diagram of the overall implementation process of steps S301 and S302 in the diagram;

[0032] Figure 7 for Figure 2 The illustrated embodiment is a schematic diagram of a specific implementation process for segmenting lesions on the lung image;

[0033] Figure 8 for Figure 2 The illustrated embodiment is a schematic diagram of the overall implementation process for segmenting lesions on the lung image;

[0034] Figure 9 for Figure 2 The illustrated embodiment is a schematic diagram of a specific implementation process for detecting the location of the lesion.

[0035] Figure 10 for Figure 2 A schematic diagram illustrating the overall process of detecting the location of the lesion in the illustrated embodiment;

[0036] Figure 11 for Figure 2 A schematic diagram of an implementation process of step S203 in another embodiment;

[0037] Figure 12 A schematic diagram illustrating the overall implementation process of lesion fusion in the lesion detection method based on CT images provided in the embodiments of this application;

[0038] Figure 13 A schematic diagram of the structure of a lesion detection device based on CT images provided in an embodiment of this application;

[0039] Figure 14 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] In existing image detection schemes, image segmentation technology is usually used to segment the image first, and the target is located based on the segmentation results. Then, image recognition technology is used to determine the type of the target. This method has the problems of slow detection speed and easy to miss targets with irregular shapes.

[0042] This application improves the detection speed of lesions by simultaneously segmenting and detecting the location of lesions on lung images, and by fusing the segmentation and detection results to obtain the location and size of the lesion in the chest CT image, thereby improving the detection accuracy of the lesion.

[0043] See Figure 1This application provides an embodiment of a lesion detection method based on CT images. This method can be applied to computer devices, such as desktop computers, servers, and other non-mobile computer devices that can process data, or mobile computer devices such as smartphones, tablets, laptops, and robots that can process data. Figure 1 As shown, the method includes:

[0044] S101. Process the chest CT image to obtain the lung image;

[0045] Specifically, a chest CT scan can be taken to obtain the subject's chest CT image, from which lung images, including the left and right lungs, can be extracted.

[0046] S102. Perform lesion segmentation on the lung image to obtain the segmentation results corresponding to each lesion region in the lung image, and simultaneously perform lesion detection on the lung image to obtain the detection results of each lesion region.

[0047] The segmentation result includes the first location information of the segmentation points contained in each lesion region within the lung image, as well as the first size data of each lesion region. The detection result may include the second location information of the detection points contained in each lesion region within the lung image, as well as the second size data of each lesion region.

[0048] In this step, the segmentation of the lesion and the location detection of the lesion are performed synchronously based on the above lung images, thus improving the processing speed.

[0049] S103. The segmentation results and detection results are fused to obtain the location and size of each lesion region in the chest CT image.

[0050] Specifically, by using a preset lesion discrimination model and lesion segmentation model, the lesion segmentation results and lesion detection results of the lung image can be fused together. The fusion result is the location and size of each lesion region in the chest CT image.

[0051] In this embodiment, on the one hand, based on the lung image obtained by processing the chest CT image, lesion segmentation is performed on the lung image to obtain the segmentation results corresponding to each lesion region in the lung image, and lesion detection is performed on the lung image simultaneously to obtain the detection results of each lesion region, which can improve the speed of lesion detection. On the other hand, by fusing the lesion segmentation results and detection results, the position and size of each lesion region on the chest CT image are obtained. Since the final output position and size of the lesion region are obtained by fusing the synchronously acquired lesion segmentation results and lesion detection results, the two processing results are complementary, thus improving the accuracy and efficiency of lesion detection.

[0052] See Figure 2 Another embodiment of this application provides a lesion detection method based on CT images. This method can be applied to computer devices, such as desktop computers, servers, and other non-mobile computer devices that can process data, or mobile computer devices such as smartphones, tablets, laptops, and robots that can process data. Figure 2 As shown, the method includes:

[0053] S201. Using a preset lung parenchyma extraction model, extract a specific image from a chest CT image and use the specific image as a lung image, wherein the specific image is matched to an outer bounding box containing the left lung region and the right lung region.

[0054] Specifically, a chest CT scan of the subject is obtained through CT imaging. Then, a specific image is extracted from the CT image using a preset lung parenchyma extraction model. This specific image serves as the lung image and is matched with an outer bounding box containing the left and right lung regions.

[0055] Optionally, to further improve the accuracy of lesion segmentation and lesion detection results, in another embodiment of this application, such as... Figure 3 As shown, step S201 can also be replaced with:

[0056] S301. Using a preset lung parenchyma extraction model, extract the specific image from the chest CT image. The specific image is matched to the bounding box that includes the left lung region and the right lung region.

[0057] S302. Input the specific image into the preset lung segmentation model for processing to obtain the lung image output by the lung segmentation model.

[0058] In another embodiment of this application, such as Figure 4 As shown, step S201 or S301 may specifically include the following steps:

[0059] S401. Using a pre-trained three-dimensional convolutional neural network, extract the lung parenchyma from the chest CT image to obtain the left lung category voxel and the right lung category voxel.

[0060] S402. Calculate the maximum connected region of the left lung category voxel and the right lung category voxel, and expand the preset pixel region based on the maximum connected region to obtain the outer bounding box containing the left lung region and the right lung region.

[0061] S403. Map the coordinate system of the bounding box to the spatial coordinate system of the chest CT image to obtain the specific image matching the bounding box after removing background noise.

[0062] As is understandable, a voxel is a "volume element," and each voxel represents a spatial volume in a three-dimensional mesh, usually a unit cube.

[0063] Specifically, firstly, the acquired CT images of the lungs (i.e., the raw CT data) are preprocessed. This preprocessing includes setting a HU (Hounsfield unit) window for the CT image. For example, the range of the HU window can be [-1200, 600]. The numerical range of the HU window can be set according to actual needs, and this specification does not impose any limitations on it. Then, based on the HU window, the HU values ​​of the CT image are normalized to pixel values. HU is a commonly used dimensionless unit in CT, used for standard and convenient expression of CT values. Specifically, it can be obtained by linearly transforming the measured attenuation coefficient.

[0064] Then, the preprocessed CT image is downsampled (Resize↓) to the first value of 128px. 3 Here, px represents pixels. The first value can be set according to actual needs. Lung parenchyma is extracted using a pre-trained lung parenchyma extraction model, resulting in voxels for the left lung, right lung, and background (i.e., parts excluding the left and right lungs). This lung parenchyma extraction model is preferably a lightweight 3D convolutional neural network, such as a 3D CNN (Convolutional Neural Network) model. Specifically, the downsampled CT image is input into the 3D CNN network, which segments the voxels into left lung, right lung, and background categories, thereby removing background voxels.

[0065] Furthermore, the connected components of the left lung category voxel and the right lung category voxel are calculated, and the largest connected component among all connected components is retained. This largest connected component is then expanded outward by a preset pixel region to obtain an outer bounding box containing the left and right lung regions, thereby roughly extracting, as shown in the figure. Figure 5 The size of the preset pixel regions for the left and right lung regions shown can be set according to actual needs, or the maximum connected region can be left unexpanded; this manual does not limit this. The external bounding box is as follows: Figure 5 A 3D cuboid.

[0066] Furthermore, by transforming the coordinate system of the bounding box into the spatial coordinate system of the CT image, a specific image matching the bounding box with background noise removed can be obtained. Specifically, the coordinates of the bounding box can be mapped to the spatial coordinate system of the CT image (original image) to obtain the image in the CT image corresponding to the bounding box, that is, to obtain the specific image in the CT image with background noise removed.

[0067] Understandably, in CT images, the outer portion of the lung (i.e., the outer background) is easily misidentified as a lung lobe or lung nodule, as it blends seamlessly with the lung interior, which is both black and white. This can interfere with the lung lobe segmentation and lesion detection process. In this application, after processing with the aforementioned lung parenchyma extraction model, the region corresponding to the bounding box is extracted from the CT image. Since the extracted bounding box region only contains the lung parenchyma, interference from the outer portion of the CT image can be effectively avoided.

[0068] Furthermore, in step S302, the specific image with background noise removed is segmented using a pre-trained lung segmentation model, and the lung image output by the lung segmentation model is obtained.

[0069] Understandably, the specific image with background noise removed can be input into the lung lobe segmentation model for processing. The lung lobe segmentation model performs lung lobe segmentation on the specific image, obtaining a specific image containing the lung lobe segmentation results. This specific image containing the lung lobe segmentation results can be used as a lung image obtained by processing a CT image. The lung lobe segmentation results are used to indicate the regions in the specific image corresponding to the right upper lobe, right middle lobe, right lower lobe, left upper lobe, and left lower lobe, respectively. Alternatively, the specific image with background noise removed can be first divided into left and right lung images, and then the left and right lung images can be input into the lung lobe segmentation model for processing separately. By dividing it into left and right lung images, the image size input to the lung lobe segmentation model is reduced, improving the processing speed of the lung lobe segmentation model. This specification does not limit this approach.

[0070] Specifically, the left and right lung images are first segmented from the specific image after background noise removal. Each of these images is then downsampled to a first value and input into a pre-trained lung segmentation model. This model performs lung segmentation on the left and right lung images, resulting in segmented left and right lung images. Next, the segmented left and right lung images are inversely upsampled (Resize↑) to restore their original image size before downsampling. The upsampled left and right lung images are then pasted into a zero-based matrix of the same size as the CT image, yielding the final lung segmentation result, i.e., the lung image. This lung image is now divided into corresponding lung lobe categories. Downsampling the left and right lung images reduces their size, thereby reducing the computational load of the lung segmentation model and improving its processing efficiency. Of course, it is also possible not to perform downsampling and upsampling operations on the left and right lung images, and this manual does not limit this.

[0071] The overall implementation process of steps S301 and S302 above can also be found in [reference needed]. Figure 6 The lung lobe segmentation model can be, for example, a 3D deep convolutional network. The difference between the lung lobe segmentation model and the lung parenchyma extraction model is that the lung parenchyma extraction model is a shallow convolutional network model with approximately 5-6 layers, while the lung lobe segmentation model is a deep convolutional network model with up to ten layers. Furthermore, the objectives of the lung parenchyma extraction model and the lung lobe segmentation model differ. The former performs binary segmentation (e.g., background and lung parenchyma categories) or tri-class segmentation (e.g., left lung category, right lung category, and background category), while the latter performs multi-class segmentation (e.g., five lung lobe categories plus background classification).

[0072] In this embodiment, the processing basis for subsequent operations such as lesion segmentation, lesion detection, and fusion of lesion segmentation and lesion detection results performed in steps S202 and S203 can be either the lung image obtained in step S201 or the lung image divided into different lung lobe categories obtained in step S302. For ease of description, the following embodiments take the processing basis of lesion segmentation and lesion detection based on the lung image obtained in step S201 as an example. The specific process of performing lesion segmentation, lesion detection, and fusion of lesion segmentation and lesion detection results based on the lung image obtained in step S302 can be referred to in the following embodiments, and will not be repeated here.

[0073] If subsequent operations such as lesion segmentation and lesion detection are performed based on the lung images divided into different lung lobe categories obtained in step S302, after determining the location and size of each lesion region in the chest CT image, the lung lobe category to which each lesion region belongs can be determined by combining the lung lobe segmentation results obtained in step S302. The lung lobe category can include the upper lobe of the right lung, the middle lobe of the right lung, the lower lobe of the right lung, the upper lobe of the left lung, or the lower lobe of the left lung, etc. Therefore, doctors and other operators do not need to determine the lung lobe category to which each lesion region belongs based on experience, which helps doctors and other operators to accurately judge the patient's condition and improve the accuracy and efficiency of diagnosis.

[0074] S202. Perform lesion segmentation on the lung image to obtain the segmentation results corresponding to each lesion region in the lung image, and simultaneously perform lesion detection on the lung image to obtain the detection results of each lesion region.

[0075] The segmentation of lesions on the lung image to obtain segmentation results corresponding to each lesion region within the lung image can include: segmenting the lung image using a pre-trained lesion segmentation model to obtain the segmentation results, which include the first position information of the segmentation points contained in each lesion region within the lung image and the first size data of each lesion region. A segmentation point is a point that determines the location of a lesion region. The segmentation point can be set according to actual needs; for example, when the lesion region is circular, the segmentation point can be the center of the lesion region, and the first size data can be the radius or diameter of the lesion region, etc. Each lesion region can be a regular shape such as a circle, quadrilateral, or triangle, and the segmentation point can be the center point of that lesion region.

[0076] Specifically, multiple cubic sub-regions are extracted from the lung image according to a preset three-dimensional overlapping sliding extraction algorithm;

[0077] The lung image is subjected to three-dimensional overlapping sliding extraction according to a preset sliding window size and a preset step size to obtain multiple cubic sub-regions;

[0078] The lung image can be divided into blocks according to the preset sliding window size and preset step size to obtain several cubic sub-regions. The preset sliding window size and preset step size can be set as needed, and this manual does not limit them.

[0079] Specifically, such as Figure 8 As shown, a lung image extracted from a chest CT image is obtained; this lung image is the region of interest (ROI) image. This lung image is then downsampled to a second value, for example, 1 mm. 3Where mm represents millimeters. Then, the downsampled lung image is divided into blocks using a shift window prediction algorithm. Specifically, cubic sub-regions are extracted from the downsampled lung image according to a preset window size and step size, where the preferred window size is 128px. 3 The preferred step size is 64px. 3 The extracted cube sub-region has a size of 128px. 3 .

[0080] Each of the cubic sub-regions is input into a pre-trained lesion segmentation model to segment the lesions and obtain the segmentation result. The segmentation result includes the first location information of the segmentation points contained in each lesion region in the lung image and the first size data of each lesion region.

[0081] Specifically, see Figure 7 The process of inputting each cubic sub-region into a pre-trained lesion segmentation model to segment lesions and obtain the segmentation result may include the following steps:

[0082] S2021A: Segment each cube sub-region using the lesion segmentation model and output the probability map corresponding to each cube sub-region;

[0083] The extracted cubic sub-regions are sequentially input into the pre-trained lesion segmentation model. This order can be either the extraction order of the cubic sub-regions or the set numbering order.

[0084] In this embodiment, the lesion segmentation model takes the lung nodule segmentation model as an example. The lung nodule segmentation model is, for example, 3D-UNet, 3D-VNet, or 3D nn-UNet. The lesion segmentation model outputs a probability map for each cubic sub-region (i.e., each sliding window). The probability map is a segmentation image containing probability values, which refer to the probability value that a single cubic sub-region belongs to the lesion region.

[0085] S2021B: The probability maps corresponding to each cube sub-region are stitched together to obtain the probability map corresponding to the lung image.

[0086] The probability maps corresponding to each cubic sub-region are reassembled in this order to obtain the probability map corresponding to the lung image.

[0087] If there is an overlap between adjacent cube sub-regions during splicing, the probability values ​​of the overlapping parts in the adjacent cube sub-regions are averaged or weighted to obtain the probability value of the overlapping part.

[0088] S2021C. Process the probability map corresponding to the lung image according to the first preset probability threshold to determine the segmentation result.

[0089] The regions in the probability map whose probability values ​​are higher than the first preset probability threshold are identified as lesion regions, and the first location information of the segmentation points contained in each lesion region and the first size data of each lesion region are obtained.

[0090] Specifically, such as Figure 8 As shown, the cube sub-regions and overlapping regions whose probability values ​​in the probability map corresponding to the lung image exceed the first preset probability threshold are identified as lesion regions. An image containing each lesion region is output, and the image corresponding to the lesion region is backsampled to restore it to the size of the lung image, thereby determining the first position information of the segmentation points contained in each lesion region in the lung image and the first size data of each lesion region.

[0091] Furthermore, lesion detection is performed on the lung image, and the detection results for each lesion region can include:

[0092] The lung image is processed into multiple three-dimensional predicted lesion regions, and the detection results of each lesion region in the lung image are determined based on each three-dimensional predicted lesion region.

[0093] The lung image processing includes multiple three-dimensional predicted lesion regions, including:

[0094] The lung image is processed into multiple two-dimensional slices, and lesions are detected in these multiple two-dimensional slices using a preset two-dimensional detection model to obtain information on the predicted lesion areas contained in each two-dimensional slice.

[0095] Based on the information of the predicted lesion area, a three-dimensional reconstruction is performed to obtain the corresponding three-dimensional predicted lesion area.

[0096] Specifically, see Figure 9 The detection of lesions in this lung image, and the results of the detection of each lesion region, mainly include the following steps:

[0097] S2022A: Process the lung image into multiple two-dimensional slices;

[0098] S2022B: Detect lesions in the multiple two-dimensional slices using a preset two-dimensional detection model to obtain information on the predicted lesion areas contained in each two-dimensional slice.

[0099] Multiple two-dimensional slices are combined into multiple input images, wherein each pair of two-dimensional slices with a preset number of layers is combined into one input image, and each input image is input into the two-dimensional detection model.

[0100] Specifically, the two-dimensional slice is synthesized into multiple input images with multiple channels, preferably three channels. The two-dimensional detection model can be, but is not limited to, the YOLOv5 model or the Faster R-CNN model.

[0101] The information of the predicted lesion region includes the position information of the edge points of the predicted lesion region, specifically the position coordinates of each vertex of the predicted lesion region. The predicted lesion region is a 2D (two-dimensional) square region, and the coordinate position of each pixel on the edge of the 2D square region or the corresponding coordinate position of the 2D square region is a known quantity. Of course, the information of the predicted lesion region may also include the position information of points inside the predicted lesion region or the size data of the predicted lesion region, etc., and this specification does not limit this.

[0102] S2022C. Based on the information of the predicted lesion area, perform three-dimensional reconstruction to obtain the corresponding three-dimensional predicted lesion area.

[0103] Based on the information of the predicted lesion areas contained in the multiple two-dimensional slices obtained from the input image, the corresponding three-dimensional predicted lesion areas are obtained according to the preset three-dimensional reconstruction algorithm. Specifically, based on the edge point position information of the predicted lesion area, the three-dimensional edge point position information corresponding to the edge point position information is calculated, and the predicted lesion area is reconstructed in three dimensions according to the three-dimensional edge point position information to obtain multiple three-dimensional predicted lesion areas.

[0104] S2022D: Based on each three-dimensional predicted lesion region, determine the detection results of each lesion region within the lung image.

[0105] Specifically, three-dimensional predicted lesion areas with a probability value of not less than the second preset probability threshold that belong to the true yang category are selected, and the selected three-dimensional predicted lesion areas are determined as target lesion areas.

[0106] Alternatively, select three-dimensional predicted lesion areas with a probability value of not less than the second preset probability threshold that belong to the true yang category, and determine the third preset probability threshold based on the probability value of each selected three-dimensional predicted lesion area. Then, select the three-dimensional predicted lesion areas with a probability value of not less than the third preset probability threshold that belong to the true yang category as the target lesion areas.

[0107] In one embodiment, by inputting the three-dimensional predicted lesion region into a pre-trained three-dimensional classification model, the probability value of each three-dimensional predicted lesion region belonging to the true positive category can be obtained. Then, based on a second preset probability threshold, three-dimensional predicted lesion regions with a probability value of belonging to the true positive category not less than the second preset probability threshold are designated as target lesion regions, while three-dimensional predicted lesion regions with a probability value of belonging to the true positive category less than the second preset probability threshold are classified as false positive categories, thereby excluding three-dimensional predicted lesion regions belonging to the false positive category. The position coordinates of the target lesion region are used as the position coordinates of the corresponding lesion region in the lung image to obtain the detection result. The preferred three-dimensional classification model is the Efficient-Net network model. The second preset probability threshold is determined based on the training process of the three-dimensional classification model. Alternatively, the probability value of each three-dimensional predicted lesion region belonging to the true positive category can be obtained directly using the three-dimensional classification model; or, the probability value of each three-dimensional predicted lesion region belonging to the false positive category can be obtained using the three-dimensional classification model, and then the probability value of each three-dimensional predicted lesion region belonging to the true positive category can be calculated based on the probability value belonging to the false positive category. This specification does not limit this approach.

[0108] In another embodiment, by inputting the three-dimensional predicted lesion region into a pre-trained three-dimensional classification model, the probability value of each three-dimensional predicted lesion region belonging to the true positive category can be obtained. First, based on a second preset probability threshold, three-dimensional predicted lesion regions with probability values ​​not less than the second preset probability threshold are selected. Then, based on the corresponding probability values ​​of each selected three-dimensional predicted lesion region, a third preset probability threshold is determined. Finally, based on the third preset probability threshold, the previously selected three-dimensional predicted lesion regions with probability values ​​not less than the third preset probability threshold belonging to the true positive category are determined as target lesion regions. The third preset probability threshold can be determined based on the average or weighted sum of the probability values ​​corresponding to each selected three-dimensional predicted lesion region. Further processing of the three-dimensional predicted lesion regions selected based on the second preset probability threshold can improve the accuracy of the determined target lesion regions, significantly reduce the possibility of misjudgment or misclassification, and further reduce the number of determined target lesion regions. Specifically, as... Figure 10As shown, the 3D classification model is "3D version of the classification model Efficient-Net". We can obtain the probability values ​​of each 3D predicted lesion region output by this "3D version of the classification model Efficient-Net" belonging to the true positive category. First, based on the second preset probability threshold, we can filter out 3D predicted lesion regions with probability values ​​not less than the second preset probability threshold, filtering out a large number of 3D predicted lesion regions belonging to the false positive category. Then, we calculate the average probability value corresponding to the filtered 3D predicted lesion regions, and determine the third preset probability threshold based on the calculated average value. Finally, based on the third preset probability threshold, we determine the previously filtered 3D predicted lesion regions with probability values ​​not less than the third preset probability threshold belonging to the true positive category as the target lesion regions.

[0109] On the one hand, by using a two-dimensional detection model, lesions can be detected on two-dimensional slices, thus having the advantage of high speed. On the other hand, by using a three-dimensional classification model to exclude three-dimensional predicted lesion areas belonging to the false positive category, a better false positive exclusion effect can be obtained. Furthermore, based on each three-dimensional predicted lesion area, the detection results of each lesion area in the three-dimensional lung image can be determined.

[0110] S203. The segmentation results and detection results are fused to obtain the location and size of each lesion region in the chest CT image.

[0111] Specifically, in combination Figure 11 and Figure 12 Step S203 may specifically include the following steps:

[0112] S2031. Based on the segmentation result, the first location information of the segmentation points of each lesion region in the lung image and the first size data of each lesion region are obtained.

[0113] In this embodiment, a circular lesion area is used as an example; other lesion areas of similar shape can be treated similarly. The dividing point can be the center point of the circular lesion area;

[0114] The first location information of the segmentation point refers to the coordinates of the center point of each lesion region in the lung image obtained after segmentation, and the first size data refers to the diameter of each lesion region obtained after segmentation.

[0115] Combination Figure 12First, based on the segmentation results obtained from the above lesion segmentation, the bounding boxes of each connected region in the lung image are obtained. Each connected region corresponds to a lesion region. Then, the parameters of the corresponding lesion region can be calculated based on the bounding box of any connected region. These parameters include the coordinates of the segmentation point of the lesion region and the diameter of the lesion region. The coordinates of each lesion region can be represented by the three-dimensional coordinates of the segmentation point, and the diameter is represented by d. Specifically, it can be the first segmented lesion region (z1, y1, x1, d1), the second segmented lesion region (z2, y2, x2, d2), and the third segmented lesion region (z3, y3, x3, d3), etc.

[0116] S2032. Based on the detection results, the second location information of the detection points in each lesion area and the second size data of each lesion area are obtained;

[0117] The second location information of the detection point refers to the coordinates of the center point of each lesion region in the lung image obtained after detection, and the second size data refers to the diameter of each lesion region obtained after detection.

[0118] Based on the coordinate location regions of the lesions obtained from the above lesion detection results, the coordinates of the center point of each detected lesion are calculated as the location of each detected lesion region. The coordinates of the center point are represented by three-dimensional coordinates, and the diameter of each detected lesion region is represented by d'. Specifically, the first detected lesion region is (z1', y1', x1', d1'), the second detected lesion region is (z2', y2', x2', d2'), and the third detected lesion region is (z3', y3', x3', d3'), etc.

[0119] S2033. Based on the first position information and the second position information, the segmentation point and the detection point are fused to obtain a point group, and the fused position information and fused size data of each position point in the point group are obtained.

[0120] The fused location information refers to the coordinates of each point in the point group, and the fused size data is the diameter of the point group.

[0121] Calculate the Euclidean distance between each segmentation point and each detection point. Merge segmentation points and detection points whose Euclidean distance is less than a preset distance (preferably 5 mm) into a single location point within a point group. Use the average coordinates of several segmentation points and several detection points merged into a single location point as the coordinates of that location point in the point group, and the average diameter of the lesion area to which several segmentation points and several detection points belong as the diameter corresponding to that location point in the point group. Alternatively, the weighted sum of the coordinates of several segmentation points and several detection points can be used as the coordinates of that location point, and the weighted sum of the diameters of the lesion areas to which several segmentation points and several detection points belong can be used as the diameter corresponding to that location point; this specification does not limit this approach. Specifically, assuming the Euclidean distance between the segmentation point of the first segmented lesion region and the detection point of the first detected lesion region is less than a preset distance, the first segmented lesion region and the first detected lesion region need to be fused. The resulting fusion location information and fusion size data are (z1”, y1”, x1”, d1”), where z1” is the average of z1 and z1’, y1” is the average of y1 and y1’, x1” is the average of x1 and x1’, and d1” is the average of d1 and d1’. Assuming the Euclidean distance between the segmentation point of the second segmented lesion region and the detection point of the second detected lesion region is less than a preset distance, the second segmented lesion region and the second detected lesion region need to be fused. The resulting fusion location information and fusion size data are (z2”, y2”, x2”, d2”), where the calculation process of z2”, y2”, x2”, and d2” is similar to that described above and will not be repeated here.

[0122] S2034. Based on the fusion location information and fusion size data of each location point in the point group, extract the three-dimensional sub-regions corresponding to each location point from the image of the lesion, and use the extracted three-dimensional sub-regions to identify false positives through the preset lesion discrimination model.

[0123] The preset lesion discrimination model can be a pre-trained 3D efficientNet model.

[0124] Three-dimensional sub-regions corresponding to each location point are extracted based on the fused size data. For example, this three-dimensional sub-region can be a cube. The three-dimensional sub-region is then used to identify lesions using a pre-trained lesion discrimination model to obtain the probability value of each three-dimensional sub-region belonging to a lesion region. This three-dimensional sub-region can be a cube, calculated with the diameter of each location point in the point group as the side length and the location point as the center point. Of course, the three-dimensional sub-region can also be other shapes, and this specification does not limit this.

[0125] S2035. Based on the three-dimensional sub-regions that exclude false positives, obtain the location and size of each lesion region in the chest CT image.

[0126] Optionally, three-dimensional sub-regions with a probability value less than a preset threshold belonging to the lesion area are classified as false positives and excluded. Three-dimensional sub-regions with a probability value greater than the preset threshold are retained. These retained sub-regions are then directly input into a preset fine-grained lesion segmentation model, such as 3D-Unet, to obtain finely segmented lesion areas corresponding to each sub-region. The finely segmented lesion areas can present the specific outlines of each lesion area. The difference between the fine-grained lesion segmentation model and the aforementioned lesion segmentation model lies in the parameter settings; the fine-grained model can achieve finer granularity and better segmentation results. Processing the retained three-dimensional sub-regions with the fine-grained lesion segmentation model yields accurate outlines of each lesion area, which is beneficial for the accuracy of the final position and size of each lesion area in the chest CT image.

[0127] A mask image corresponding to the finely segmented lesion region can be obtained, and this mask image can be mapped onto the chest CT image, essentially padding the mask image back to the original chest CT image size. Based on the mapping result, the parameters of each final lesion in the final lesion segmentation image of the chest CT image can be determined, including the center coordinates and diameter of each final lesion. Furthermore, adjacent connected components in the mapping result can be fused to make the obtained final lesion region more accurate.

[0128] Optionally, the retained three-dimensional sub-regions can be directly input into a preset fine lesion segmentation model to obtain the finely segmented lesion regions corresponding to each three-dimensional sub-region. Specifically, this may include: determining the diameter of the final lesion through the fine lesion segmentation model, and obtaining the specific outline of the finely segmented lesion regions corresponding to each three-dimensional sub-region based on the determined diameter of the final lesion.

[0129] Optionally, to further improve the accuracy of the detection results, in another embodiment of this application, the retained three-dimensional sub-region may not be directly input into the preset fine lesion segmentation model. Instead, based on the segmentation mask of the three-dimensional sub-region that excludes false positives and the diameter of the point group in the three-dimensional sub-region that excludes false positives, it is determined whether to input the three-dimensional sub-region that excludes false positives into the fine lesion segmentation model for segmentation. This can reduce the training difficulty of the fine lesion segmentation model and improve its processing efficiency. If the three-dimensional sub-region that excludes false positives contains the segmentation mask and the diameter of the point group in the three-dimensional sub-region that excludes false positives is smaller than the first diameter, or if the three-dimensional sub-region that excludes false positives does not contain the segmentation mask and the diameter of the point group in the three-dimensional sub-region that excludes false positives is larger than the second diameter, then the three-dimensional sub-region that excludes false positives is input into the fine lesion segmentation model for segmentation.

[0130] Specifically, it is first determined whether the retained 3D sub-region contains a segmentation mask. This segmentation mask is the mask in the segmented image of the lesion region obtained based on the segmentation results of the lesion region mentioned above. Specifically, if the location points within the 3D sub-region are obtained by fusing segmentation points and detection points, then the 3D sub-region contains the corresponding segmentation mask; if the location points within the 3D sub-region are obtained by fusing only segmentation points, then the 3D sub-region contains the corresponding segmentation mask; and if the location points within the 3D sub-region are obtained by fusing only detection points, then the 3D sub-region does not have a corresponding segmentation mask.

[0131] like Figure 12 As shown, if the retained three-dimensional sub-region contains a segmentation mask, it is determined whether the diameter of the corresponding point group is smaller than the preset first diameter (e.g., 16mm). If it is not smaller than the preset first diameter, the segmentation mask is used to obtain the first lesion region without the need for processing by the fine lesion segmentation model, thus simplifying the processing. If it is smaller than the preset first diameter, the retained three-dimensional sub-region is input into the preset fine lesion segmentation model to obtain the second lesion region.

[0132] If the retained 3D sub-region does not contain a segmentation mask, it is determined whether the diameter of the corresponding point group is greater than a preset second diameter (e.g., 8 mm), where the second diameter is smaller than the first diameter. In this embodiment, the fine lesion segmentation model can also be used only to obtain the finely segmented lesion regions corresponding to each 3D sub-region. In this case, if the diameter of the point group is not greater than the preset second diameter, a new mask is constructed according to the size of the 3D sub-region, and a third lesion region is obtained based on the constructed mask. If the diameter of the point group is greater than the preset second diameter, the retained 3D sub-region can be input into the aforementioned preset fine lesion segmentation model to obtain a fourth lesion region.

[0133] The mask images corresponding to the first, second, third, and fourth lesion regions are then obtained and mapped onto the chest CT image. This means the mask images are padded back to the original chest CT image size. Based on the mapping result, the parameters of each final lesion in the final lesion segmentation image of the chest CT image can be determined, including the center coordinates and diameter of each final lesion. Furthermore, adjacent connected components in the mapping result can be fused to make the obtained final lesion regions more accurate. The mask image corresponding to the first lesion region is the aforementioned segmentation mask, and the mask image corresponding to the third lesion region is the newly constructed mask.

[0134] Therefore, the aforementioned processing does not require inputting every retained 3D sub-region into the fine lesion segmentation model for processing, which simplifies the processing and significantly improves the efficiency of obtaining mask images. Furthermore, different methods of obtaining lesion regions are used for 3D sub-regions that do not contain segmentation masks and have different diameters of corresponding point groups, which can achieve personalized processing for different retained 3D sub-regions and improve the accuracy of the final lesion obtained.

[0135] Optional, such as Figure 12As shown in the lower right corner, if the retained 3D sub-region does not contain a segmentation mask and the diameter of the corresponding point group is greater than the preset second diameter (e.g., 8mm), further refinement can be performed instead of directly inputting the retained 3D sub-region into the preset fine lesion segmentation model for processing. The refinement process is as follows: if the diameter of the point group is less than the third diameter (e.g., 32mm), the diameter of the point group in the retained 3D sub-region is adjusted to the third diameter; if the diameter of the point group is not less than the third diameter but less than the fourth diameter (e.g., 64mm), the diameter of the point group in the retained 3D sub-region is adjusted to the fourth diameter; if the diameter of the point group is greater than the fourth diameter, the diameter of the point group in the retained 3D sub-region is not adjusted. The third diameter is less than the fourth diameter. Subsequently, the updated 3D sub-region can be extracted based on the adjusted diameter, and then the updated 3D sub-region can be input into the preset fine lesion segmentation model for processing. In practice, during the refinement process, different three-dimensional sub-regions can be updated again according to the original diameter of the point group. This is beneficial for the accuracy of the final lesion location and size determined based on the updated three-dimensional sub-regions, and improves the reliability of the determined lesion location and size in chest CT images. Of course, the aforementioned first, second, third, and fourth diameters can be adjusted according to actual needs, and this specification does not impose any restrictions on this. If the diameter of the point group in any retained three-dimensional sub-region is the same as the specific diameter, the retained three-dimensional sub-region can be processed by using a method where the corresponding point group diameter is smaller than the specific diameter or a method where the corresponding point group diameter is larger than the specific diameter. This specification does not impose any restrictions on this, where the specific diameter can be the first, second, third, or fourth diameter.

[0136] Furthermore, after obtaining the location and size of the lesion in the chest CT image, the location and size of the lesion in the chest CT image can also be output according to a preset output method. For example, a detection report containing text information and / or image information of the location and size of the lesion in the chest CT image can be generated and displayed on the screen, or navigation information can be generated and saved based on the location and size of the lesion in the chest CT image, etc.

[0137] In this embodiment, on the one hand, based on the lung image obtained by processing the chest CT image, lesion segmentation is performed on the lung image to obtain the segmentation results corresponding to each lesion region in the lung image, and lesion detection is performed on the lung image simultaneously to obtain the detection results of each lesion region, which can improve the speed of lesion detection. On the other hand, by fusing the lesion segmentation results and detection results, the position and size of each lesion region on the chest CT image are obtained. Since the final output position and size of the lesion region are obtained by fusing the synchronously acquired lesion segmentation results and lesion detection results, the two processing results are complementary, thus improving the accuracy and efficiency of lesion detection.

[0138] See Figure 13 This application provides a schematic diagram of the structure of a lesion detection device based on CT images, according to an embodiment. For ease of explanation, only the parts relevant to the embodiment are shown. This device can be a computer device with data processing capabilities, or it can be configured as a virtual module within a computer device with data processing capabilities. Figure 13 As shown, the device includes:

[0139] Processing module 501 is used to process chest CT images to obtain lung images;

[0140] The segmentation module 502 is used to segment the lesions on the lung image to obtain segmentation results corresponding to each lesion region in the lung image;

[0141] The detection module 503 is used to simultaneously detect lesions based on the lung image and obtain the detection results of each lesion area;

[0142] The fusion module 504 is used to fuse the segmentation results and the detection results to obtain the position and size of each lesion region in the chest CT image.

[0143] The specific process by which each module implements its respective function can be found in the relevant content of the above embodiments, and will not be repeated here.

[0144] In this embodiment, on the one hand, based on the lung image obtained by processing the chest CT image, lesion segmentation is performed on the lung image to obtain the segmentation results corresponding to each lesion region in the lung image, and lesion detection is performed on the lung image simultaneously to obtain the detection results of each lesion region, which can improve the speed of lesion detection. On the other hand, by fusing the lesion segmentation results and detection results, the position and size of each lesion region on the chest CT image are obtained. Since the final output position and size of the lesion region are obtained by fusing the synchronously acquired lesion segmentation results and lesion detection results, the two processing results are complementary, thus improving the accuracy and efficiency of lesion detection.

[0145] Furthermore, the segmentation module 502 is also used to segment the lung image using a pre-trained lesion segmentation model to obtain the segmentation result, the segmentation result including the first position information of the segmentation points contained in each lesion region in the lung image and the first size data of each lesion region.

[0146] The segmentation module 502 is also used to extract multiple cubic sub-regions from the lung image according to a preset three-dimensional overlapping sliding extraction algorithm;

[0147] Each of the cubic sub-regions is input into a pre-trained lesion segmentation model to segment the lesions and obtain the segmentation results.

[0148] The segmentation module 502 is further configured to segment each of the cubic sub-regions using the lesion segmentation model and output a probability map corresponding to each of the cubic sub-regions;

[0149] The probability maps corresponding to each of the cube sub-regions are stitched together to obtain the probability map corresponding to the lung image;

[0150] The probability map corresponding to the lung image is processed according to a first preset probability threshold to determine the segmentation result.

[0151] The detection module 503 is also used to process the lung image into multiple three-dimensional predicted lesion regions, and determine the detection results of each lesion region in the lung image based on each three-dimensional predicted lesion region.

[0152] The detection module 503 is also used to process the lung image into multiple two-dimensional slices, and to perform lesion detection on the multiple two-dimensional slices through a preset two-dimensional detection model to obtain information on the predicted lesion area contained in each two-dimensional slice.

[0153] Based on the information of the predicted lesion area, a three-dimensional reconstruction is performed to obtain the corresponding three-dimensional predicted lesion area.

[0154] The information of the predicted lesion area includes the location information of the edge points of the predicted lesion area;

[0155] The detection module 503 is further configured to obtain the three-dimensional edge point position information corresponding to the edge point position information according to the edge point position information of the predicted lesion area and according to the preset three-dimensional reconstruction algorithm, and to perform three-dimensional reconstruction of the predicted lesion area according to the three-dimensional edge point position information to obtain multiple three-dimensional predicted lesion areas.

[0156] The detection module 503 is also used to obtain the probability value of each three-dimensional predicted lesion region belonging to the true positive category by inputting the three-dimensional predicted lesion region into a pre-trained three-dimensional classification model.

[0157] Based on the probability values ​​of each three-dimensional predicted lesion region belonging to the true positive category, the target lesion region is determined, and the position coordinates of the lesion in the target lesion region are used as the position coordinates of the corresponding lesion region in the lung image to obtain the detection result.

[0158] The detection module 503 is also used to filter out three-dimensional predicted lesion areas with a probability value of not less than a second preset probability threshold that belong to the true positive category, and to determine the filtered three-dimensional predicted lesion areas as the target lesion areas.

[0159] Alternatively, three-dimensional predicted lesion areas with a probability value of not less than the second preset probability threshold are selected, and a third preset probability threshold is determined based on the probability value of each selected three-dimensional predicted lesion area. The selected three-dimensional predicted lesion areas with a probability value of not less than the third preset probability threshold are then identified as target lesion areas.

[0160] The fusion module 504 is further configured to obtain, based on the segmentation results, the first location information of the segmentation points of each lesion region in the lung image and the first size data of each lesion region;

[0161] Based on the detection results, the second location information of the detection points in each lesion area and the second size data of each lesion area are obtained;

[0162] Based on the first location information and the second location information, the segmentation point and the detection point are fused to obtain a point group, and the fused location information and fused size data of each point in the point group are obtained;

[0163] Based on the fused location information and fused size data of each location point in the point group, a three-dimensional sub-region corresponding to each location point is extracted from the image of the lesion, and the extracted three-dimensional sub-region is used to identify false positives through a preset lesion discrimination model.

[0164] Based on the three-dimensional sub-regions that exclude false positives, the location and size of each lesion region in the chest CT image are obtained.

[0165] The fusion module 504 is further configured to merge the segmentation point and the detection point whose Euclidean distance is less than a preset distance into a single location point within the point group, and to use the average coordinates of the segmentation point and the detection point as the coordinates of the point group, and the average diameter as the diameter of the point group.

[0166] The segmentation results include segmentation masks corresponding to each lesion region;

[0167] The fusion module 504 is also used to determine whether to input the three-dimensional sub-region that excludes false positives into the fine lesion segmentation model for segmentation based on the segmentation mask of the three-dimensional sub-region that excludes false positives and the diameter of the point group in the three-dimensional sub-region that excludes false positives.

[0168] If the three-dimensional sub-region excluding false positives contains the segmentation mask, and the diameter of the point group in the three-dimensional sub-region excluding false positives is smaller than the first diameter, or if the three-dimensional sub-region excluding false positives does not contain the segmentation mask, and the diameter of the point group in the three-dimensional sub-region excluding false positives is larger than the second diameter, then the three-dimensional sub-region excluding false positives is input into the fine lesion segmentation model for segmentation.

[0169] The segmentation results output by the fine lesion segmentation model are used to obtain the location and size of each lesion region in the chest CT image.

[0170] The processing module 501 is further configured to extract a specific image from the chest CT image using a preset lung parenchyma extraction model, and use the specific image as the lung image, wherein the specific image is matched to an outer bounding box containing the left lung region and the right lung region.

[0171] The processing module 501 is also used to extract a specific image from the chest CT image using a preset lung parenchyma extraction model, wherein the specific image is matched to an outer bounding box containing the left lung region and the right lung region.

[0172] The specific image is input into a preset lung lobe segmentation model for processing to obtain a lung image output by the lung lobe segmentation model.

[0173] The specific process by which each module implements its respective function can be found in the relevant content of the above embodiments, and will not be repeated here.

[0174] See Figure 14 The present application provides a schematic diagram of the hardware structure of an electronic device according to an embodiment. Figure 14 As shown, the electronic device 10 includes a memory 11 and a processor 12.

[0175] The memory 11 stores an executable computer program 13. The processor 12, coupled to the memory 11, calls the executable computer program 13 stored in the memory and executes it as follows: Figures 1 to 12 The above embodiments provide lesion detection methods based on CT images.

[0176] For example, the computer program 13 can be divided into one or more modules / units, which are stored in the memory 11 and executed by the processor 12 to complete the present invention. The one or more modules / units may include various modules in the CT image-based lesion detection device in the above embodiments, such as: processing module 501, segmentation module 502, detection module 503, and fusion module 504.

[0177] Furthermore, the device also includes:

[0178] At least one input device and at least one output device.

[0179] The processor 12, memory 11, input devices, and output devices mentioned above can be connected via a bus.

[0180] The input device can be a camera, touch panel, physical buttons, or mouse, etc. The output device can be a display screen.

[0181] Furthermore, the device may include more components than illustrated, or combine certain components, or different components, such as network access devices, sensors, etc.

[0182] Processor 12 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0183] The memory 11 can be, for example, a hard disk drive, non-volatile memory (such as flash memory or other electronically programmable erasure-restricted memory used to form a solid-state drive), volatile memory (such as static or dynamic random access memory), etc., and this application embodiment is not limited thereto. Specifically, the memory 11 can be an internal storage unit of the electronic device, such as the hard disk or RAM of the electronic device. The memory 11 can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device. Further, the memory 11 can include both internal storage units and external storage devices of the electronic device. The memory 11 is used to store computer programs and other programs and data required by the terminal. The memory 11 can also be used to temporarily store data that has been output or will be output.

[0184] Furthermore, embodiments of this application also provide a computer-readable storage medium, which may be disposed in the electronic device of the above embodiments, and the computer-readable storage medium may be the aforementioned... Figure 14 The memory 11 in the illustrated embodiment may, for example, be a non-transitory computer-readable storage medium. This computer-readable storage medium stores a computer program that, when executed by a processor, implements the aforementioned functionality. Figures 1 to 12 The lesion detection methods based on CT images are described in the various embodiments shown. Furthermore, the computer storage medium can also be any medium capable of storing program code, such as a USB flash drive, external hard drive, read-only memory (ROM), RAM, magnetic disk, or optical disk.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0186] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0188] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0189] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0190] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0191] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0192] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0193] The above is a description of the lesion detection method, apparatus, electronic device and computer-readable storage medium based on CT images provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there will be changes in the specific implementation and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A lesion detection method based on CT images, characterized in that, include: Chest CT images are processed to obtain lung images; The lung image is segmented to obtain segmentation results corresponding to each lesion region within the lung image, and lesion detection is performed on the lung image simultaneously to obtain detection results for each lesion region; The segmentation results and the detection results are fused to obtain the location and size of each lesion region in the chest CT image; The step of detecting lesions in the lung images to obtain detection results for each lesion region includes: The lung image is processed into multiple two-dimensional slices, and lesions are detected in the multiple two-dimensional slices using a preset two-dimensional detection model to obtain the edge point location information of the predicted lesion area contained in each two-dimensional slice. Based on the edge point location information of the predicted lesion area, the three-dimensional edge point location information corresponding to the edge point location information is obtained according to the preset three-dimensional reconstruction algorithm, and the predicted lesion area is reconstructed in three dimensions according to the three-dimensional edge point location information to obtain multiple three-dimensional predicted lesion areas. Based on each three-dimensional predicted lesion region, the detection results of each lesion region within the lung image are determined.

2. The method according to claim 1, characterized in that, The step of segmenting the lung image to obtain segmentation results corresponding to each lesion region within the lung image includes: The lung image is segmented using a pre-trained lesion segmentation model to obtain the segmentation result, which includes the first location information of the segmentation points contained in each lesion region within the lung image and the first size data of each lesion region.

3. The method according to claim 2, characterized in that, The step of segmenting lesions in the lung image using a pre-trained lesion segmentation model to obtain the segmentation result includes: According to the preset three-dimensional overlapping sliding extraction algorithm, multiple cubic sub-regions are extracted from the lung image; Each of the cubic sub-regions is input into a pre-trained lesion segmentation model to segment the lesions and obtain the segmentation results.

4. The method according to claim 3, characterized in that, The step of inputting each of the cubic sub-regions into a pre-trained lesion segmentation model for lesion segmentation and obtaining the segmentation result includes: Each of the cubic sub-regions is segmented using the lesion segmentation model, and a probability map corresponding to each of the cubic sub-regions is output. The probability maps corresponding to each of the cube sub-regions are stitched together to obtain the probability map corresponding to the lung image; The probability map corresponding to the lung image is processed according to a first preset probability threshold to determine the segmentation result.

5. The method according to claim 1, characterized in that, The detection results for determining each lesion region within the lung image based on each three-dimensional predicted lesion region include: By inputting the three-dimensional predicted lesion regions into a pre-trained three-dimensional classification model, the probability value of each three-dimensional predicted lesion region belonging to the true yang category is obtained; Based on the probability values ​​of each three-dimensional predicted lesion region belonging to the true positive category, the target lesion region is determined, and the position coordinates of the target lesion region are used as the position coordinates of the corresponding lesion region in the lung image to obtain the detection result.

6. The method according to claim 5, characterized in that, The step of determining the target lesion region based on the probability values ​​of each of the three-dimensional predicted lesion regions belonging to the true positive category includes: Three-dimensional predicted lesion areas with a probability value of not less than the second preset probability threshold belonging to the true yang category are selected, and the selected three-dimensional predicted lesion areas are determined as the target lesion areas; Alternatively, three-dimensional predicted lesion areas with a probability value of not less than the second preset probability threshold are selected, and a third preset probability threshold is determined based on the probability value of each selected three-dimensional predicted lesion area. The selected three-dimensional predicted lesion areas with a probability value of not less than the third preset probability threshold are then identified as target lesion areas.

7. The method according to claim 1, characterized in that, The process of fusing the segmentation results and the detection results to obtain the location and size of each lesion region in the chest CT image includes: Based on the segmentation results, the first location information of the segmentation points of each lesion region in the lung image and the first size data of each lesion region are obtained; Based on the detection results, the second location information of the detection points in each lesion area and the second size data of each lesion area are obtained; Based on the first location information and the second location information, the segmentation point and the detection point are fused to obtain a point group, and the fused location information and fused size data of each point in the point group are obtained; Based on the fused location information and fused size data of each location point in the point group, a three-dimensional sub-region corresponding to each location point is extracted from the image of the lesion, and the extracted three-dimensional sub-region is used to identify false positives through a preset lesion discrimination model. Based on the three-dimensional sub-regions that exclude false positives, the location and size of each lesion region in the chest CT image are obtained.

8. The method according to claim 7, characterized in that, The step of fusing the segmentation points and the detection points according to the first location information and the second location information to obtain a point group, and obtaining the fused position information and fused size data of each point in the point group, includes: The segmentation points and the detection points whose Euclidean distance is less than a preset distance are merged into a single location point within the point group. The average coordinates of the segmentation points and the detection points are used as the coordinates of the point group, and the average diameter is used as the diameter of the point group.

9. The method according to claim 7, characterized in that, The process of obtaining the location and size of each lesion region in the chest CT image based on the three-dimensional sub-regions excluding false positives includes: Based on the segmentation mask of the three-dimensional sub-region that excludes false positives and the diameter of the point group in the three-dimensional sub-region that excludes false positives, it is determined whether to input the three-dimensional sub-region that excludes false positives into the fine lesion segmentation model for segmentation; If the three-dimensional sub-region excluding false positives contains the segmentation mask, and the diameter of the point group in the three-dimensional sub-region excluding false positives is smaller than the first diameter, or if the three-dimensional sub-region excluding false positives does not contain the segmentation mask, and the diameter of the point group in the three-dimensional sub-region excluding false positives is larger than the second diameter, then the three-dimensional sub-region excluding false positives is input into the fine lesion segmentation model for segmentation. The location and size of each lesion region in the chest CT image are obtained based on the segmentation results output by the fine lesion segmentation model.

10. The method according to claim 1, characterized in that, The process of processing chest CT images to obtain lung images includes: A specific image is extracted from the chest CT image using a preset lung parenchyma extraction model, and the specific image is used as the lung image, wherein the specific image is matched with an outer bounding box containing the left lung region and the right lung region.

11. The method according to claim 1, characterized in that, The process of processing chest CT images to obtain lung images includes: A specific image is extracted from the chest CT image using a preset lung parenchyma extraction model, wherein the specific image is matched to an outer bounding box containing the left lung region and the right lung region; The specific image is input into a preset lung lobe segmentation model for processing to obtain a lung image output by the lung lobe segmentation model.

12. A lesion detection device based on CT images, characterized in that, include: The processing module is used to process chest CT images to obtain lung images; The segmentation module is used to segment the lung image into lesions, and obtain segmentation results corresponding to each lesion region in the lung image; The detection module is used to simultaneously detect lesions in the lung images and obtain the detection results for each lesion region; The fusion module is used to fuse the segmentation results and the detection results to obtain the position and size of each lesion region in the chest CT image; The detection module is further configured to process the lung image into multiple two-dimensional slices, and perform lesion detection on the multiple two-dimensional slices using a preset two-dimensional detection model to obtain the edge point location information of the predicted lesion region contained in each two-dimensional slice. Based on the edge point location information of the predicted lesion region, a preset three-dimensional reconstruction algorithm is used to obtain the three-dimensional edge point location information corresponding to the edge point location information. The predicted lesion region is then reconstructed in three dimensions according to the three-dimensional edge point location information to obtain multiple three-dimensional predicted lesion regions. Finally, based on each three-dimensional predicted lesion region, the detection result of each lesion region in the lung image is determined.

13. An electronic device, characterized in that, include: Memory and processor; The memory stores executable computer programs; The processor coupled to the memory invokes the executable computer program stored in the memory to perform each step of the lesion detection method based on CT images as described in any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the lesion detection method based on CT images as described in any one of claims 1-11.

Images