3D medical image anomaly detection and classification method

Abstract

The invention relates to the technical field of medical anomaly detection, in particular to a 3D medical image anomaly detection and classification method. The method comprises the following steps: acquiring 3D medical image data for a target part; converting the 3D medical image data to obtain a 2D slice sequence, and if two adjacent 2D slices have an abnormal region, determining an intersection-to-union ratio; when the intersection-to-union ratio is greater than a preset intersection-to-union ratio threshold value, judging that the abnormal regions of the two adjacent 2D slices are the same abnormal region; taking the 2D slices with the same abnormal area in the X 2D slices as target slices; determining a classification result of each target slice; and determining the anomaly type of the same anomaly region according to each classification result. According to the method, the 2D slice sequence is analyzed, the accuracy of detection and classification is improved, meanwhile, the 3D medical image with large calculation and analysis amount does not need to be analyzed, and therefore the calculation amount can be reduced.

Description

Technical Field

[0001] The present application relates to the technical field of medical anomaly detection, and in particular to a 3D medical image anomaly detection and classification method. Background Art

[0002] In modern medical diagnostics, 3D medical imaging technologies (such as 3D CT, 3D MRI, 3D X-rays, and 3D ultrasound) have become key tools for early disease detection and diagnosis, thanks to their ability to visualize the three-dimensional structure of internal tissues and organs. For example, in lung tumor screening, 3D CT images can accurately capture the morphological changes of millimeter-level nodules, providing a basis for early intervention. In neurosurgery planning, 3D MRI images can clearly demonstrate the spatial relationship between brain blood vessels and lesions, significantly improving surgical success rates.

[0003] However, existing 3D medical imaging anomaly detection technologies still have significant limitations. Traditional methods based on manual feature extraction and machine learning rely on medical experts to manually design features, such as extracting texture features based on gray-level co-occurrence matrices and segmenting lesions using region growing algorithms. However, these features have limited expressive power, resulting in a high rate of false positives. Summary of the Invention

[0004] The embodiments of the present application provide a 3D medical image anomaly detection and classification method, which can solve at least one of the above technical problems.

[0005] According to one aspect of an embodiment of the present application, a 3D medical image anomaly detection and classification method is proposed, the 3D medical image anomaly detection and classification method comprising: Acquire 3D medical imaging data of the target area; Converting the 3D medical image data to obtain a 2D slice sequence related to the target part, wherein the 2D slice sequence includes X 2D slices, where X ≥ 2; If there are abnormal areas between the two adjacent 2D slices, determining the intersection-over-union ratio of the abnormal areas between the two adjacent 2D slices; When the IoU is greater than a preset IoU threshold, determining that the abnormal regions of the two adjacent 2D slices are the same abnormal region; The 2D slice having the same abnormal area among the X 2D slices is used as the target slice; Determining the classification results of each of the target slices; The abnormality type of the same abnormal area is determined according to each of the classification results.

[0006] In the above solution, converting the 3D medical image data to obtain a 2D slice sequence related to the target part includes: Continuously slicing the 3D medical image data along a preset slicing direction according to a preset slice thickness to obtain the 2D slice sequence including X 2D slices, each of the 2D slices having the preset slice thickness; The preset segmentation direction is one or more of a direction corresponding to the horizontal plane, a direction corresponding to the coronal plane, and a direction corresponding to the sagittal plane.

[0007] In the above solution, the 2D slice includes multiple pixels, and whether the 2D slice has an abnormal area is determined by the following method: Determining a pixel value of each pixel in the 2D slice; For each pixel point, if the pixel value of the pixel point is greater than a preset pixel threshold, the pixel point is determined to be an abnormal pixel point; If the pixel values ​​corresponding to the respective pixel points of the target area of ​​the M consecutive 2D slices are all greater than the preset pixel threshold, the target area is determined to be an abnormal area and abnormal areas are determined to exist in all the M consecutive 2D slices.

[0008] In the above solution, determining the intersection-over-union ratio of abnormal regions of two adjacent 2D slices includes: Determining the abnormal region of the current 2D slice as a first abnormal region; determining the abnormal region of the previous 2D slice or the abnormal region of the next 2D slice as a second abnormal region; Determine the coordinates of a first upper left corner and a first lower right corner of the first abnormal area, and determine the coordinates of a second upper left corner and a second lower right corner of the second abnormal area; Determine an intersection area and a union area according to the first upper left corner coordinate, the first lower right corner coordinate, the second upper left corner coordinate, and the second lower right corner coordinate; The intersection area is divided by the union area to obtain the intersection-to-union ratio.

[0009] In the above solution, determining the abnormality type of the same abnormal area according to each of the classification results includes: If there are K classification results that are consistent and the confidence levels of the K classification results are all greater than a preset confidence threshold, the abnormality types corresponding to the K classification results are used as the abnormality types of the same abnormal area.

[0010] In the above solution, before continuously slicing the 3D medical image data along a preset slicing direction according to a preset slice thickness, the method further includes: The 3D medical image data is normalized and subjected to noise elimination processing.

[0011] Beneficial effects of the present application: The present application can convert 3D medical imaging data into a 2D slice sequence that is easy to analyze, and analyze the 2D slices in the 2D slice sequence. Therefore, through dimensionality reduction analysis, the misjudgment rate of abnormality detection in the target area can be reduced.

[0012] Furthermore, if there are abnormal areas in two adjacent 2D slices, the intersection and union ratio is used to determine whether the abnormal areas in the two adjacent 2D slices are the same abnormal area, thereby avoiding the analysis obstacles caused by different abnormal areas and enabling the analysis results of the same abnormal area to effectively classify the abnormal type.

[0013] In summary, the present application improves the accuracy of detection and classification by analyzing 2D slice sequences. At the same time, there is no need to analyze 3D medical images, which require large amounts of calculation and analysis, thereby reducing the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a system architecture diagram of the 3D medical image anomaly detection and classification method provided in the embodiments of the present application; Figure 2 A flowchart of a 3D medical image anomaly detection and classification method provided in an embodiment of the present application; Figure 3 A logical diagram of a 3D medical image anomaly detection and classification method provided in an embodiment of the present application; Figure 4 A schematic diagram of the detection logic of a 2D model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of this application.

[0016] It should be noted that some of the processes described in the specification, claims, and the above-mentioned drawings include multiple steps that appear in a specific order, but it should be clearly understood that these steps can be executed in a different order than that in which they appear in this document or in parallel. The step numbers are only used to distinguish between different steps, and the numbers themselves do not represent any order of execution. In addition, descriptions such as "first," "second," or "target" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. "Multiple" in this document refers to at least two.

[0017] It is worth noting that in the specific implementation of the present application, when medical imaging data and other related data are involved, when the above embodiments of the present application are applied to specific products or technologies, it is necessary to obtain the permission or consent of the target object, and the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards. For example, when the embodiments of the present application need to obtain medical imaging data and other related data, the target object's separate permission or separate consent can be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the target object's separate permission or separate consent, the medical imaging data used to enable the normal operation of the embodiments of the present application can be obtained.

[0018] See also Figure 1 , Figure 1 1 is a diagram of the system architecture used in the 3D medical image anomaly detection and classification method provided in the embodiment of the present application, which includes a terminal 140, an Internet 130, a gateway 120, a server 110, etc.

[0019] Terminal 140 can take various forms, including desktop computers, laptops, PDAs (personal digital assistants), mobile phones, in-vehicle terminals, and dedicated terminals. Furthermore, it can be a single device or a combination of multiple devices. For example, multiple desktop computers connected via a local area network, sharing a common display and working collaboratively, collectively constitute terminal 140. Terminal 140 can communicate with Internet 130 via wired or wireless means to exchange data.

[0020] Server 110 is a computer system that provides certain services to terminal 140. Compared to ordinary terminal 140, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines). Server 110 can also communicate with the Internet 130 via wired or wireless means to exchange data.

[0021] Gateway 120, also known as a gateway or protocol converter, implements network interconnection at the transport layer and is a computer system or device that performs a conversion function. It acts as a translator between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateways can also provide filtering and security functions. Messages sent from terminal 140 to server 110 are sent through gateway 120 to the corresponding server 110. Messages sent from server 110 to terminal 140 are also sent through gateway 120 to the corresponding terminal 140.

[0022] The following is a detailed introduction to the specific implementation of the embodiment of this application: See also Figure 2 , Figure 2 3D medical image anomaly detection and classification method provided in an embodiment of the present application is a flow chart. The 3D medical image anomaly detection and classification method can be implemented by the server 110 and / or the terminal 140. Figure 2 The 3D medical image anomaly detection and classification method shown includes: Step 210: Acquire 3D medical imaging data of the target part; Step 220: convert the 3D medical image data to obtain a 2D slice sequence related to the target part, wherein the 2D slice sequence includes X 2D slices, where X≥2; Step 230: If there is an abnormal area between two adjacent 2D slices, determine the intersection-over-union ratio of the abnormal areas between the two adjacent 2D slices; Step 240: When the IoU is greater than a preset IoU threshold, determining that the abnormal regions of the two adjacent 2D slices are the same abnormal region; Step 250: Use the 2D slice with the same abnormal area among the X 2D slices as the target slice; Step 260: Determine the classification results of each target slice; Step 270: Determine the abnormality type of the same abnormal area according to each of the classification results.

[0023] The following is a detailed explanation of the complete embodiment of the present application in conjunction with steps 210-270: This application proposes a method to reduce the dimensionality of 3D medical imaging data to a 2D slice sequence, thereby improving the accuracy of anomaly detection and reducing the computational complexity of analysis.

[0024] In step 210 , the target part may be a brain, chest or other parts, and the 3D medical imaging data may be MRI, CT, ultrasound or other related medical imaging data.

[0025] In step 220 , the 3D medical image data is converted to obtain a 2D slice sequence related to the target part, wherein the 2D slice sequence includes X 2D slices, where X≥2.

[0026] This application proposes a 2D model that converts 3D medical imaging data to produce a 2D slice sequence associated with the target area. The 2D model, known as a 2D slice image abnormality detection model, is trained using traditional methods (such as thresholding and filtering techniques) or deep learning-based methods (such as convolutional neural networks (CNNs) and Transformer networks). This 2D model can also be used for subsequent abnormality detection.

[0027] In one embodiment, converting the 3D medical image data to obtain a 2D slice sequence related to the target site includes: performing normalization and noise elimination processing on the 3D medical image data; Continuously slicing the 3D medical image data along a preset slicing direction according to a preset slice thickness to obtain the 2D slice sequence including X 2D slices, each of the 2D slices having the preset slice thickness; The preset segmentation direction is one or more of a direction corresponding to the horizontal plane, a direction corresponding to the coronal plane, and a direction corresponding to the sagittal plane.

[0028] The direction corresponding to the horizontal plane is the direction along the horizontal plane, the direction corresponding to the coronal plane is the direction along the coronal plane, and the direction corresponding to the sagittal plane is the direction along the sagittal plane. Since the horizontal, coronal, and sagittal planes are orthogonal planes known in the art, they are not illustrated here. The preset slice thickness can be set as needed, for example, 2 mm, 3 mm, etc.

[0029] In step 230, if abnormal regions exist in two adjacent 2D slices, the intersection-over-union ratio (IoU) of the abnormal regions in the two adjacent 2D slices is determined to determine whether the abnormal regions in the two adjacent 2D slices are the same abnormal region. This allows for subsequent analysis of the same abnormal region to be meaningful. It should be noted that the same abnormal region indicates that the approximate locations of the two adjacent 2D slices are both marked as abnormal. For example, if the current abnormal region is A and the abnormal region of the previous 2D slice is B, the areas of A and B may not be the same and may vary. Therefore, the same abnormal region indicates that an abnormality has occurred in a certain approximate area of ​​the two adjacent 2D slices.

[0030] In one embodiment, the 2D slice includes a plurality of pixels, and whether the 2D slice has an abnormal area is determined by: Determining a pixel value of each pixel in the 2D slice; For each pixel point, if the pixel value of the pixel point is greater than a preset pixel threshold, the pixel point is determined to be an abnormal pixel point; If the pixel values ​​corresponding to the respective pixel points of the target area of ​​the M consecutive 2D slices are all greater than the preset pixel threshold, the target area is determined to be an abnormal area and abnormal areas are determined to exist in all the M consecutive 2D slices.

[0031] Threshold segmentation (such as Otsu threshold method, adaptive threshold method) and morphological operations (such as opening and closing operations, contour extraction) are used to detect anomalies in 2D slices.

[0032] Deep learning detection method: 2D slices are input into an improved lightweight 2D CNN model (such as YOLOv8 and Faster R-CNN) for abnormal area detection.

[0033] Post-processing optimization: Morphological post-processing is performed on the detected abnormal areas. Small noise areas are removed through connected area analysis and area threshold filtering. At the same time, the boundary smoothing algorithm is used to optimize the contour boundaries of the abnormal areas.

[0034] Only when a spatially correlated abnormal region is detected in M ​​consecutive slices (M ≥ 2) is the region considered a true abnormality.

[0035] In one embodiment, determining the abnormal region of the current 2D slice as a first abnormal region; determining the abnormal region of the previous 2D slice or the abnormal region of the next 2D slice as a second abnormal region; Determine the coordinates of a first upper left corner and a first lower right corner of the first abnormal area, and determine the coordinates of a second upper left corner and a second lower right corner of the second abnormal area; Determine an intersection area and a union area according to the first upper left corner coordinate, the first lower right corner coordinate, the second upper left corner coordinate, and the second lower right corner coordinate; The intersection area is divided by the union area to obtain the intersection-to-union ratio.

[0036] Specifically, assume that the abnormal regions of two adjacent 2D slices are rectangular regions A (the first abnormal region) and B (the second abnormal region) respectively. The coordinates of A are represented as (xA1, yA1, xA2, yA2), where (xA1, yA1) is the first upper-left corner coordinate of rectangular region A, and (xA2, yA2) is the first lower-right corner coordinate of rectangular region A; the coordinates of rectangular region B are represented as (xB1, yB1, xB2, yB2), where (xB1, yB1) is the second upper-left corner coordinate of rectangular region B, and (xB2, yB2) is the second lower-right corner coordinate of rectangular region B. Calculate the coordinates of the intersection rectangle: The upper-left corner coordinate of the intersection rectangle is (max(xA1, xB1), max(yA1, yB1)), briefly denoted as (xinter1, yinter1), and the lower-right corner coordinate is (min(xA2, xB2), min(yA2, yB2)), briefly denoted as (xinter2, yinter2).

[0037] Calculate the intersection area: If the upper-left corner coordinate (xinter1, yinter1) and the lower-right corner coordinate (xinter2, yinter2) of the intersection rectangle satisfy xinter1 < xinter2 and yinter1 < yinter2, then the intersection area is Sintersection = (xinter2 - xinter1) * (yinter2 - yinter1); if the above conditions are not met, that is, the two rectangular regions have no intersection, then Sintersection = 0.

[0038] Calculate the union area: The area of rectangular region A is SA = (xA2 - xA1) * (yA2 - yA1), the area of rectangular region B is SB = (xB2 - xB1) * (yB2 - yB1), and the union area Sunion = SA + SB - Sintersection.

[0039] Calculate the intersection over union: IoU = Sintersection / Sunion, where IoU is the intersection over union.

[0040] For example, the coordinates of the abnormal region A detected in two adjacent slices are (100, 100, 200, 200), and the coordinates of the abnormal region B are (120, 120, 220, 220), where (100, 100) is the first upper-left corner coordinate of the abnormal region A, (200, 200) is the first upper-right corner coordinate of the abnormal region A, (120, 120) is the second upper-left corner coordinate of the abnormal region B, and (220, 220) is the second upper-right corner coordinate of the abnormal region B.

[0041] Calculate the coordinates of the intersection rectangle. The coordinates of the upper left corner are (max (100, 120), max (100, 120))=(120,120), and the coordinates of the lower right corner are (min (200, 220), min (200, 220))=(200, 200). Calculate the intersection area, Sintersection = (200 - 120) × (200 - 120) = 6400; Calculate the union area, SA=(200 - 100)×(200 - 100)=10000, SB=(220 - 120)×(220 -120)=10000, Sunion = 10000 + 10000 - 6400 = 13600; Calculate the IoU value, IoU = 6400 / 13600 ≈ 0.47. Since the preset IoU threshold TioU is 0.5, it is not considered to be the same abnormal area at this time. In step 240 , when the IoU is greater than a preset IoU threshold, it is determined that the abnormal regions of the two adjacent 2D slices are the same abnormal region.

[0042] Specifically, the preset intersection-over-union (IoU) threshold is set to 0.5, that is, when the IoU is greater than 0.5, the abnormal regions of the two adjacent 2D slices are considered to be the same abnormal region.

[0043] In step 250 , the 2D slices with the same abnormal region among the X 2D slices are taken as target slices, that is, the 2D slices with the same abnormal region are obtained to perform abnormality type analysis.

[0044] In step 260, these 2D slices are input into a pre-trained classification network (such as ResNet50, VisionTransformer) for abnormality type classification to obtain the classification results and corresponding confidence levels.

[0045] In step 270, if there are K consistent classification results and the confidence levels of these K classification results are all greater than a preset confidence threshold, the abnormality types corresponding to these K classification results are used as the abnormality types of the same abnormal region. In other words, the abnormal region is finally classified as a corresponding type (e.g., tumor, inflammation, cyst, etc.) only when the classification results in at least K (K ≥ 2) 2D slices are consistent and the classification confidence levels all exceed a preset confidence threshold (e.g., 0.7).

[0046] The following is a detailed introduction to the specific application implementation environment of this application: The 3D medical image anomaly detection and classification method of this application includes the following: 1. 3D Medical Image Preprocessing and 2D Slice Conversion Data standardization: 3D medical imaging data (including but not limited to 3D CT, 3D MRI, 3D ultrasound, etc.) are converted into a unified format. Image data from different devices and different scanning parameters are converted into a unified data format and normalized to ensure that all 3D medical imaging data have a consistent value range.

[0047] Noise elimination: Adaptive filtering algorithms (such as non-local mean filtering and bilateral filtering) combined with deep learning noise reduction models (such as DnCNN and RED-Net) are used to denoise image data, effectively removing random noise and equipment artifacts while preserving image detail features.

[0048] Multi-plane 2D slice generation: 3D medical imaging data is sliced ​​continuously along three orthogonal planes—the horizontal, coronal, and sagittal—at preset slice thicknesses to generate a 2D slice sequence. The spatial resolution of each slice is uniformly adjusted to the preset value using a bicubic interpolation algorithm to ensure consistency in subsequent processing.

[0049] Expert annotation and training set construction: The generated 2D slices are annotated with abnormal areas. The annotation information includes the location, boundary, and type of the abnormal area to build a high-quality training dataset.

[0050] 2. Detection of abnormal regions in 2D slices Traditional image processing detection methods are used: threshold segmentation (such as Otsu threshold method, adaptive threshold method) and morphological operations (such as opening and closing operations, contour extraction) are used to detect anomalies in 2D slices.

[0051] Deep learning detection method: 2D slices are input into an improved lightweight 2D CNN model (such as YOLOv8 and Faster R-CNN) for abnormal area detection.

[0052] Post-processing optimization: Morphological post-processing is performed on the detected abnormal areas. Small noise areas are removed through connected area analysis and area threshold filtering. At the same time, the boundary smoothing algorithm is used to optimize the contour boundaries of the abnormal areas.

[0053] 3. Abnormal Region Localization Reasoning in 3D Medical Images Multi-slice correlation detection mechanism: When localizing abnormal regions in 3D medical images, a sliding window strategy is adopted to jointly analyze N (N≥2) consecutive 2D slices. For example, if the preset pixel threshold is set to 150, and the pixel value of a pixel point is greater than 150, then it is considered an abnormal pixel point.

[0054] Spatio-temporal continuity verification: Only when abnormal regions related to spatial positions are detected in at least M (M≥2) consecutive slices, is this region determined to be a true abnormality. The spatial position correlation is determined by calculating the intersection over union (IoU) of abnormal regions in adjacent slices.

[0055] 4. Method for determining spatial correlation of abnormal regions For whether there are abnormal regions in the same region of two consecutive slices, it is judged according to the IoU value of the intersection over union. A threshold is set according to the image quality. For example, if IoU = 0.5, when IoU is greater than 0.5, it is judged that there is an abnormality. The following is an example: Suppose the abnormal regions of two adjacent slices are rectangular regions A and B. The coordinates of A are represented as (xA1, yA1, xA2, yA2), where (xA1, yA1) is the upper left corner coordinate of the rectangle, and (xA2, yA2) is the lower right corner coordinate; the coordinates of the rectangular region B are represented as (xB1, yB1, xB2, yB2), where (xB1, yB1) is the upper left corner coordinate of the rectangle, and (xB2, yB2) is the lower right corner coordinate; Calculate the coordinates of the intersection rectangle: The upper left corner coordinate of the intersection rectangle is (max(xA1, XB1), max(yA1, yB1)), briefly recorded as (xinter1, yinter1), and the lower right corner coordinate is (min(xA2, xB2), min(yA2, yB2)), briefly recorded as (xinter2, yinter2).

[0056] Calculate the intersection area: If the upper left corner coordinate (xinter1, yinter1) and the lower right corner coordinate (xinter2, yinter2) of the intersection rectangle satisfy xinter1 < xinter2 and yinter1 < yinter2, then the intersection area Sintersection = (xinter2 - xinter1) × (yinter2 - yinter1); if the above conditions are not met, that is, the two rectangles have no intersection, then Sintersection = 0.

[0057] Calculate the area of ​​the union: the area of ​​rectangle A is SA = (xA2-xA1) x (yA2-yA1), the area of ​​rectangle B is SB = (xB2-xB1) x (yB2-yB1), and the area of ​​the union is Sunion = SA + SB - Sintersection.

[0058] Calculate the intersection over union ratio: IoU=Sintersection / Sunion.

[0059] 5. Abnormality classification and diagnosis Multi-slice consistency classification rule: For abnormal areas verified by spatial continuity, their feature information in all relevant slices is collected and input into a pre-trained classification network (such as ResNet50, Vision Transformer) for abnormality type classification.

[0060] Classification confidence fusion: The abnormal area is finally classified as the corresponding type (such as tumor, inflammation, cyst, etc.) only when the classification results in at least K (K ≥ 2) slices are consistent and the classification confidence exceeds the preset threshold Tc2 (such as 0.7).

[0061] The specific flow chart is as follows Figure 3 As shown, 3D medical images are converted into 2D slices for analysis, and abnormal areas are detected and classified. Figure 4 This is a schematic diagram of the 2D detection model during detection.

[0062] The following is a detailed description of the technical solution of this application with reference to specific examples. Figure 3 and 4 As shown, so that those skilled in the art can better understand and implement the present application. Example 1: Abnormality Detection in 3D Brain MRI Images 1. 3D Medical Image Preprocessing and 2D Slice Conversion Data standardization processing: Obtain brain 3D MRI image data from a hospital, and the original data format is NIfTI.

[0063] Noise elimination processing: The non-local mean filtering algorithm is used to denoise the image data. The search window size is set to 7×7, the similarity window size is set to 3×3, and the filter strength parameter is set to 10 to effectively remove Gaussian noise in the image. Multi-planar 2D slice generation: 3D MRI images were sliced ​​along the transverse, coronal, and sagittal planes with a slice thickness of 1 mm. All slices were resized to a uniform spatial resolution of 512 × 512 pixels using the OpenCV library's resize function via bicubic interpolation. Expert annotation and training set construction: Three neurosurgery experts were invited to annotate the abnormal areas of the generated 2D slices. The annotation information included abnormality types such as tumors and hemorrhages, and finally a training dataset containing 5,000 annotated slices was constructed. 2. Detection of abnormal regions in 2D slices Traditional preprocessing: 2D slices are segmented using the Otsu threshold method to extract the outlines of possible abnormal areas; then morphological opening operations (structural element size is 3×3) are used to remove small noisy areas, and closing operations (structural element size is 5×5) are used to fill internal holes. Deep learning network anomaly detection: The preprocessed slices are input into the improved YOLOv8 model, which embeds the CBAM module in the backbone network. The model is trained using the Adam optimizer with a learning rate set to 0.001. The model is trained for 200 epochs and achieves a detection accuracy of 92% on the validation set. Post-processing optimization: Connected region analysis is performed on the detected abnormal areas, areas with an area smaller than 50 pixels are removed, and the contours of the abnormal areas are smoothed using the approxPolyDP function of OpenCV. 3. Abnormal Region Localization Reasoning in 3D Medical Images A sliding window strategy was used, selecting three consecutive axial slices for joint analysis each time, with the initial detection confidence threshold Tc1 set to 0.4. When a region was detected to appear in two consecutive slices with similar coordinates, its intersection over union (IoU) was calculated. 4. Method for determining spatial correlation of abnormal regions Assume that the coordinates of abnormal region A detected in two adjacent slices are (100, 100, 200, 200), and the coordinates of abnormal region B are (120, 120, 220, 220): Calculate the coordinates of the intersection rectangle. The coordinates of the upper left corner are (max (100, 120), max (100, 120))=(120,120), and the coordinates of the lower right corner are (min (200, 220), min (200, 220))=(200, 200). Calculate the intersection area, Sintersection = (200 - 120) × (200 - 120) = 6400; Calculate the union area, SA=(200 - 100)×(200 - 100)=10000, SB=(220 - 120)×(220 -120)=10000, Sunion = 10000 + 10000 - 6400 = 13600; Calculate the IoU value, IoU = 6400 / 13600 ≈ 0.47. Since the preset IoU threshold TioU is 0.5, they are not considered to be the same abnormal area at this time. 5. Abnormality classification and diagnosis The slices associated with the abnormal region that passed spatial continuity verification were input into a pre-trained ResNet50 classification network for type determination. If the abnormal region was determined to be a tumor type in at least three slices and the classification confidence level exceeded 0.8, the abnormal region was ultimately classified as a tumor. Example 2: Abnormality Detection in Lung 3D CT Images 1. 3D Medical Image Preprocessing and 2D Slice Conversion Building on the original implementation, this method comprehensively upgrades the preprocessing of lung 3D CT images. The raw data, in NIfTI format, is converted into Numpy arrays using the Nibabel library. A pretrained U-Net model is then used to segment the lungs, accurately obtaining lung segmentation masks. This operation more accurately separates lung tissue than traditional thresholding methods, laying a solid foundation for subsequent processing. During the normalization step, the lung region mean was subtracted and divided by the standard deviation. HU values ​​were clipped to the range of [-500, 1500] to effectively suppress outliers. Denoising employed a composite strategy, first using bilateral filtering (spatial sigma set to 1.5 and color sigma set to 0.5) for preliminary noise reduction. This was then combined with non-local means filtering and anisotropic diffusion to remove noise while preserving edge details to the greatest extent possible. In the slice conversion process, in addition to conventional 0.8mm slice thickness along three planes, thick slice reconstruction (MIP) technology was added to generate sagittal thick slice images with a thickness of 3mm, enhancing the visibility of subtle lesions. After the resolution of all slices was uniformly adjusted to 256×256 pixels, the training set was expanded to 5,000 annotated slices through data augmentation techniques (rotation, flipping, elastic deformation, etc.) to improve model generalization capabilities. 2. Detection of abnormal regions in 2D slices 2.1 Optimization of traditional methods The traditional adaptive threshold method is combined with the Otsu algorithm for anomaly detection. The Otsu algorithm is used to automatically calculate the optimal threshold, improve the accuracy of threshold segmentation, and reduce manual intervention. At the same time, the segmentation results are post-processed through morphological opening and closing operations to eliminate small noise areas and fill holes. 2.2 Deep Learning Model Upgrade The deep learning model uses an improved Faster R-CNN architecture. The backbone network replaces the SE module with a CBAM (Convolutional Block Attention Module). CBAM focuses on both channel and spatial features, effectively extracting key information from abnormal areas. During training, a cosine annealing learning rate adjustment strategy was used, with a batch size of 32 and 200 epochs trained. Mixed-precision training techniques were also employed to accelerate convergence. On the validation set, detection accuracy increased to 93%, a significant improvement over the original model. In the post-processing stage, a combination of non-maximum suppression (NMS) and soft-NMS algorithms was used to accurately remove redundant detection boxes. 3. Abnormal Region Localization Reasoning in 3D Medical Images and Subsequent Steps To improve detection reliability, temporal information is incorporated into the inference of abnormal region location. Each slice is tested using a trained Faster R-CNN model. A valid abnormality is considered only when an abnormality is detected in the same region in at least two consecutive slices, and the overlap of the abnormal regions exceeds 60%. Furthermore, 3D spatial information is combined to calculate the three-dimensional coordinates and volume of the abnormal region, providing richer quantitative metrics for clinical diagnosis. 4. 3D Medical Image Abnormality Classification The abnormality classification module uses the Vision Transformer model, incorporating a contrastive learning strategy during the pre-training phase to enhance the model's ability to distinguish between different abnormality types. During classification, an abnormality is determined to be an inflammatory region only when at least two slices in the same region are classified as inflammatory, with a confidence level exceeding 0.75 for each slice. Furthermore, the model output provides information such as the probability distribution of the abnormal region and its boundary with normal tissue, assisting doctors in making more accurate diagnoses.

[0064] In summary, the 3D medical image anomaly detection method of the present application has the following beneficial effects: 1. Improved detection accuracy: 2D models are used to detect and classify image anomalies. Anomalies are identified only when at least two consecutive slices show anomalies. For anomaly classification, an area must have at least two slices with anomalies belonging to a specific category for that area to be classified as such, improving the accuracy of anomaly detection and classification.

[0065] 2. Reduced computational effort: 3D medical image anomaly detection and classification are achieved through 2D models, significantly reducing computational effort.

[0066] In addition, the terms "comprises" and "comprising" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or apparatus that comprises a series of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, product or apparatus.

[0067] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0068] It should be understood that in the description of the embodiments of the present application, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood to exclude the number itself, and above, below, within, etc. are understood to include the number itself.

[0069] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0070] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0071] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0072] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0073] It should also be understood that the various implementation methods provided in the embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0074] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or portion of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal. It can be implemented in whole or in part using software, hardware (such as processing circuits or memory), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the functionality of the module or unit.

[0075] The above is a specific description of the implementation methods of the present application, but the present application is not limited to the above implementation methods. Technical personnel familiar with the art can also make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A 3D medical image anomaly detection and classification method, characterized in that: The method comprises: Acquire 3D medical imaging data of the target area; Converting the 3D medical image data to obtain a 2D slice sequence related to the target part, wherein the 2D slice sequence includes X 2D slices, where X ≥ 2; If there are abnormal areas between the two adjacent 2D slices, determining the intersection-over-union ratio of the abnormal areas between the two adjacent 2D slices; When the IoU is greater than a preset IoU threshold, determining that the abnormal regions of the two adjacent 2D slices are the same abnormal region; The 2D slice having the same abnormal area among the X 2D slices is used as the target slice; Determining the classification results of each of the target slices; The abnormality type of the same abnormal area is determined according to each of the classification results.

2. The 3D medical image anomaly detection and classification method according to claim 1, characterized in that: The converting of the 3D medical image data to obtain a 2D slice sequence related to the target part includes: Continuously slicing the 3D medical image data along a preset slicing direction according to a preset slice thickness to obtain the 2D slice sequence including X 2D slices, each of the 2D slices having the preset slice thickness; The preset segmentation direction is one or more of a direction corresponding to the horizontal plane, a direction corresponding to the coronal plane, and a direction corresponding to the sagittal plane.

3. The 3D medical image anomaly detection and classification method according to claim 1, wherein: The 2D slice includes multiple pixels, and whether the 2D slice has an abnormal area is determined by the following method: Determining a pixel value of each pixel in the 2D slice; For each pixel point, if the pixel value of the pixel point is greater than a preset pixel threshold, the pixel point is determined to be an abnormal pixel point; If the pixel values ​​corresponding to the respective pixel points of the target area of ​​the M consecutive 2D slices are all greater than the preset pixel threshold, the target area is determined to be an abnormal area and abnormal areas are present in all the M consecutive 2D slices.

4. The 3D medical image anomaly detection and classification method according to claim 3, wherein: Determining the intersection-over-union ratio of abnormal areas of two adjacent 2D slices includes: Determining the abnormal region of the current 2D slice as a first abnormal region; determining the abnormal region of the previous 2D slice or the abnormal region of the next 2D slice as a second abnormal region; Determine the coordinates of a first upper left corner and a first lower right corner of the first abnormal area, and determine the coordinates of a second upper left corner and a second lower right corner of the second abnormal area; Determine an intersection area and a union area according to the first upper left corner coordinate, the first lower right corner coordinate, the second upper left corner coordinate, and the second lower right corner coordinate; The intersection area is divided by the union area to obtain the intersection-to-union ratio.

5. The 3D medical image anomaly detection and classification method according to claim 1, wherein: Determining the abnormality type of the same abnormal area according to each of the classification results includes: If there are K classification results that are consistent and the confidence levels of the K classification results are all greater than a preset confidence threshold, the abnormality types corresponding to the K classification results are used as the abnormality types of the same abnormal area.

6. The 3D medical image anomaly detection and classification method according to claim 2, wherein: Before continuously segmenting the 3D medical image data along a preset segmentation direction according to a preset slice thickness, the method further includes: The 3D medical image data is normalized and subjected to noise elimination processing.

Images