Medical image-based computer-aided diagnosis method, system and device, computer storage medium and computer program product

Abstract

The invention provides a medical image-based computer-aided diagnosis method, system and device, a computer storage medium and a computer program product. The method comprises the following steps: acquiring original DICOM data obtained based on a tomography technology; constructing three-dimensional volume data based on the original DICOM data; performing multi-plane reconstruction based on the three-dimensional volume data to generate an MPR image queue; inputting the MPR image queue into a deep learning model to obtain a multi-scale feature map corresponding to each MPR image in the MPR image queue; using a multi-plane interaction module to carry out cross-plane association fusion on the multi-scale feature map to obtain a multi-plane fused feature map; and performing global pooling and full-connection coding on the multi-plane fused feature map by using a feature fusion and coding module, and outputting a one-dimensional diagnosis vector for auxiliary diagnosis. By adopting the scheme, the original medical image data can be efficiently converted into the diagnosis vector, and automatic computer-aided diagnosis is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and in particular to a computer-aided diagnosis method, system and device based on medical images, a computer storage medium and a computer program product. BACKGROUND

[0002] In the field of medical image diagnosis, computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) have become widely used in clinical examination. A single scan usually produces a sequence of hundreds to thousands of two-dimensional tomographic images, which can constitute a three-dimensional data whole reflecting anatomical, tissue or metabolic information of the human body. However, the traditional technology adopts the way of analyzing images one by one and extracting features one by one, which faces the problems of huge data analysis and calculation, redundant feature extraction, and lack of necessary spatial position relationship between features. This leads to the fact that the existing method is difficult to quickly process sequence data, and also cannot effectively integrate the spatial correlation information between images, which restricts the efficiency and accuracy of medical image intelligent analysis, large-scale population artificial intelligence (AI) automatic screening and other applications.

[0003] Therefore, there is an urgent need for a method that can efficiently process the entire sequence and extract simplified features containing pixel value features and spatial position relationship features, in order to break through the bottleneck of traditional technology and meet the needs of fast and accurate diagnosis in clinical practice. SUMMARY

[0004] Therefore, the embodiments of the present application provide a computer-aided diagnosis method, system and device based on medical images, a computer storage medium and a computer program product, which can efficiently process medical images and output accurate diagnosis results for assisting diagnosis.

[0005] In a first aspect, embodiments of the present invention provide a computer-aided diagnostic method based on medical images, comprising the following steps: acquiring raw DICOM data obtained based on computed tomography (CT) technology; constructing three-dimensional volumetric data based on the raw DICOM data; performing multi-planar reconstruction (MPR) on the three-dimensional volumetric data to generate an MPR image queue; inputting the MPR image queue into a deep learning model, and sequentially passing it through convolutional blocks, attention modules, and feature pyramid networks of the deep learning model to obtain multi-scale feature maps corresponding to each MPR image in the MPR image queue; the deep learning model further comprising: a multi-planar interaction module and a feature fusion and encoding module; using the multi-planar interaction module to perform cross-planar correlation fusion on the multi-scale feature maps to obtain a multi-planar fused feature map with a two-dimensional structure; using the feature fusion and encoding module to perform global pooling and fully connected encoding on the multi-planar fused feature map to output a one-dimensional diagnostic vector for auxiliary diagnosis.

[0006] This approach efficiently transforms raw medical image data into diagnostic vectors, enabling automated computer-aided diagnosis. Based on a multi-module collaboration using a deep learning model, it fully extracts multi-scale and multi-planar features from medical images, effectively integrates global and local information, enhances the ability to identify complex lesions, provides doctors with quantitative diagnostic references, improves diagnostic efficiency and accuracy, and reduces the subjectivity and error of human interpretation.

[0007] In conjunction with the first aspect, in a first possible implementation of the first aspect, the MPR image queue includes at least: coronal MPR images, sagittal MPR images, and transverse MPR images; the step of performing multi-plane reconstruction (MPR) based on the three-dimensional volume data to generate the MPR image queue includes: slicing along the coronal axis, sagittal axis, and transverse axis based on the three-dimensional volume data to generate a two-dimensional image sequence in orthogonal planes, wherein the slice thickness and interval are configurable.

[0008] The aforementioned scheme specifies the composition of the MPR image queue, ensuring comprehensive anatomical perspective information for lesion observation from different angles. The configurable slice thickness and spacing allow the system to flexibly adjust image parameters according to actual diagnostic needs (such as lesion size, location, age, etc.). This enables the system to acquire high-resolution images using thin-slice scanning when observing fine structures, and to reduce data volume using thick-slice scanning when quickly viewing the overall structure, improving the flexibility and specificity of image processing, thereby optimizing the diagnostic process and enhancing adaptability to different diseases.

[0009] In conjunction with the first aspect, in the second possible implementation of the first aspect, the attention module includes: a channel attention submodule and a spatial attention submodule; the channel attention submodule implements channel feature weighting through global average pooling and a multilayer perceptron, and the spatial attention submodule implements spatial feature enhancement through convolution and a nonlinear activation function.

[0010] The above scheme employs a channel attention submodule that uses global average pooling and a multilayer perceptron to weight channel features, automatically selecting channels of significant diagnostic importance while suppressing irrelevant information. The spatial attention submodule enhances spatial features using convolution and nonlinear activation functions, focusing on the spatial region where the lesion is located. The combination of these two submodules enables the model to adaptively allocate attention resources, highlighting key diagnostic features in medical images, reducing interference from background noise and redundant information, and significantly improving the model's sensitivity and localization accuracy for lesions, thereby enhancing the accuracy and reliability of diagnosis.

[0011] In conjunction with the first aspect, in the third possible implementation of the first aspect, the feature pyramid network adopts a top-down multi-scale feature fusion strategy, performing upsampling and element-wise addition operations on the feature maps of different levels output by the convolutional block to generate multi-scale feature maps containing global structure and local details.

[0012] The aforementioned scheme effectively integrates feature maps from different levels, combining global semantic information (such as the overall morphology of organs) from high-level feature maps with local detail information (such as edge texture) from low-level feature maps to generate multi-scale feature maps that simultaneously contain rich structure and detail. This enables deep learning models to grasp both the overall anatomical structure and the detailed features of minute lesions when processing medical images, improving the model's ability to identify lesions of different sizes and shapes. It is particularly suitable for the detection and diagnosis of complex lesions, enhancing the model's generalization ability and robustness.

[0013] In conjunction with the first aspect, in the fourth possible implementation of the first aspect, the multi-plane interaction module calculates the similarity matrix between the multi-scale feature maps corresponding to each MPR image in the MPR image queue, and adjusts the attention weight values ​​of the collaborative diagnostic features based on the similarity matrix according to a preset attention weight enhancement rule.

[0014] The aforementioned scheme, by calculating a similarity matrix, can uncover the correlations between feature maps at different planes and scales, revealing key feature combinations for collaborative diagnosis. Assigning higher weights to these collaborative diagnostic features enables the model to more effectively integrate multi-plane information, comprehensively analyzing lesion features from multiple perspectives and avoiding the limitations of single-plane or single-scale information. This helps improve the diagnostic ability for complex diseases (such as lesions associated with multiple sites), providing doctors with more comprehensive and accurate diagnostic clues, and enhancing the comprehensiveness and effectiveness of diagnosis.

[0015] In conjunction with the first aspect, in the fifth possible implementation of the first aspect, the feature fusion and encoding module includes: a global average pooling layer and a fully connected layer, wherein the global average pooling layer compresses the spatial dimension of the feature map fused by the multi-plane, and the fully connected layer maps the compressed features into a one-dimensional diagnostic vector.

[0016] The above scheme uses a global average pooling layer to compress the spatial dimension of the multi-plane fused feature map. While retaining key feature information, it reduces data dimensionality and computational cost, thus lowering the risk of overfitting. The fully connected layer maps the compressed features into a one-dimensional diagnostic vector, achieving a high degree of feature abstraction and quantification, providing a concise and intuitive output for subsequent diagnostic decisions. The combination of these two layers ensures efficient generation of diagnostic vectors, making the model output easy to understand and apply. This facilitates accurate diagnostic judgments by doctors based on quantified data, improving diagnostic efficiency and clinical applicability.

[0017] Secondly, embodiments of the present invention provide a computer-aided diagnostic system based on medical images, comprising: an MPR image encoder and a deep learning module. The MPR image encoder is used to acquire raw DICOM data obtained based on tomographic scanning technology; construct three-dimensional volume data based on the raw DICOM data, and perform multi-plane reconstruction (MPR) based on the three-dimensional volume data to generate an MPR image queue; the deep learning model is used to sequentially pass the MPR image queue through its own convolutional blocks, attention modules, and feature pyramid networks to obtain multi-scale feature maps corresponding to each MPR image in the MPR image queue; the deep learning model further comprises: a multi-plane interaction module and a feature fusion and encoding module, used to perform cross-plane correlation fusion on the multi-scale feature maps using the multi-plane interaction module to obtain a two-dimensional multi-plane fused feature map; and to perform global pooling and fully connected encoding on the multi-plane fused feature map using the feature fusion and encoding module to output a one-dimensional diagnostic vector for auxiliary diagnosis. Thirdly, embodiments of the present invention provide a computer-aided diagnostic device based on medical images, the device comprising: a processor configured to execute computer-executable instructions; and a memory storing one or more computer-executable instructions, wherein when executed by the processor, the computer-executable instructions implement the various steps of the computer-aided diagnostic method based on medical images as described in the first aspect or any possible implementation thereof.

[0018] Fourthly, embodiments of the present invention provide a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the various steps of the computer-aided diagnosis method based on medical images as described in the first aspect or any possible implementation of the first aspect.

[0019] Fifthly, embodiments of the present invention provide a computer program product, including a computer program that, when executed by a processor, implements the computer-aided diagnosis method based on medical images as described in the first aspect or any possible implementation of the first aspect.

[0020] In a sixth aspect, embodiments of the present invention provide a chip system applied to an electronic device. The chip system includes one or more processors, which are configured to invoke computer instructions to cause the electronic device to perform a computer-aided diagnostic method based on medical images as described in the first aspect or any possible implementation thereof.

[0021] Understandably, the technical effects achieved by the computer-aided diagnostic system based on medical images described in the second aspect, the computer-aided diagnostic device based on medical images described in the third aspect, the computer storage medium described in the fourth aspect, the computer program product described in the fifth aspect, and the chip system described in the sixth aspect are similar to the technical effects achieved by the corresponding technical means in the computer-aided diagnostic method based on medical images described in the first aspect, and will not be repeated here. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a computer-aided diagnostic method based on medical images provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of obtaining a sequence of MRP images from three-dimensional volumetric data in one embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a sequence MRP image queue in one embodiment of the present invention; Figure 4 This is a schematic diagram of the deep learning module architecture in a computer-aided diagnostic system based on medical images, provided in an embodiment of the present invention. Figure 5This is a schematic diagram of the structure of a computer-aided diagnostic device based on medical images provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0024] It should be understood that the term "multiple" in this invention refers to two or more. In the description of this invention, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, terms such as "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply differences.

[0025] In the field of medical imaging, CT, MRI, and PET scans are commonly used clinical examination methods, with a single sequence typically containing hundreds or even thousands of two-dimensional tomographic images. Traditional methods of analyzing and extracting features image by image present challenges such as high computational demands, an excessive number of extracted features, and a lack of necessary spatial relationships between features. In intelligent medical image analysis, intelligent quality control, and label classification, especially in AI-driven automated disease screening for large populations, the ability to quickly, comprehensively, and concisely extract the overall features of the corresponding three-dimensional data from a sequence is of paramount importance. Existing technologies struggle to effectively address these issues.

[0026] This application proposes a computer-aided diagnostic method based on medical images. First, it constructs three-dimensional volumetric data from tomographic scan images. Then, it generates an MPR image queue (e.g., coronal MPR, sagittal MPR, and transverse MPR obtained by maximum density projection on three orthogonal planes: coronal, sagittal, and transverse). Next, it processes the data through a deep learning network including convolutional blocks, a Convolutional Block Attention Module (CBAM), a Feature Pyramid Network (FPN), a planar interaction module, and a feature fusion and encoding module to extract one-dimensional feature vectors containing spatial relationships. This scheme preserves three-dimensional anatomical information through orthogonal plane projection, reduces dimensionality and extracts key pixel values ​​using maximum density projection, and integrates multi-planar features through a deep learning network. It effectively solves the problems of high computational cost, feature redundancy, and lack of spatial correlation in traditional image-by-image analysis. It can quickly extract concise features, taking into account both pixel values ​​and spatial relationships, which is beneficial for improving the efficiency of intelligent medical image analysis and disease screening, and providing high-quality feature input for AI models.

[0027] Figure 1 This is a flowchart illustrating a computer-aided diagnostic method based on medical images according to one embodiment of the present invention, as shown below. Figure 1 As shown, a computer-aided diagnostic method based on medical images may include steps S101 to S106.

[0028] S101. Acquire raw DICOM data based on computed tomography (CT) technology. The raw DICOM data can originate from scans of a specified area of ​​the patient using computed tomography equipment such as CT, MRI, or PET. Each scan sequence contains a set of spatially consistent and continuous two-dimensional tomographic images. For example, a chest C-scan sequence may contain 500-1000 cross-sectional images, each carrying anatomical, tissue, or metabolic information from different layers of the same scanned area.

[0029] S102. Construct 3D volumetric data based on the original DICOM data. According to the spatial coordinate information (such as Image Position (Patient) and Image Orientation (Patient) labels) in the DICOM file, the data is sorted and integrated according to the spatial position of the scanned slices. For example, cross-sectional images are arranged in ascending order of the Z-axis coordinate, and coronal images are arranged in ascending order of the Y-axis coordinate, ensuring that adjacent slices are continuous and without misalignment in 3D space. Pixel gaps are filled using interpolation algorithms (such as cubic spline interpolation) to construct a complete 3D volumetric data cube, whose spatial resolution is consistent with the original tomographic images, reflecting the 3D distribution characteristics of human tissue.

[0030] S103. Perform multi-plane reconstruction based on three-dimensional volume data to generate an MPR image queue.

[0031] The MPR image queue includes at least: coronal MPR images, sagittal MPR images, and transverse MPR images; for ease of description, such as Figure 2 As shown, this application describes an MPR image queue comprising coronal MPR images, sagittal MPR images, and transverse MPR images as an example. In some possible implementations, the MPR image queue can be arranged according to... Figure 3 The images are arranged in the following order: coronal MPR images, sagittal MPR images, and transverse MPR images.

[0032] In some possible implementations, multi-planar reconstruction based on 3D volumetric data is used to generate an MPR image queue. This includes: slicing along the coronal, sagittal, and transverse axes based on 3D volumetric data to generate a 2D image sequence with orthogonal planes, where slice thickness and spacing are configurable. For example, slice thickness and spacing can be configured manually, automatically by the inspection equipment, or partially automatically and partially manually. For instance, parameter values ​​can be automatically determined based on a pre-trained deep learning model. In oncology radiotherapy departments using standardized scanning protocols, multiple follow-up CT scans of the same patient can use the same slice thickness parameters. In emergency rooms, when rapidly processing CT scans of patients with cerebral hemorrhage, the system can automatically select the optimal slice thickness parameters to highlight the hemorrhage area. Hybrid configuration applications include CT scans of children; for example, MPR parameters can be dynamically adjusted according to the pediatric patient's age, while allowing doctors to fine-tune them.

[0033] S104. Input the MPR image queue into the deep learning model, and sequentially pass it through the convolutional blocks, attention module, and feature pyramid network of the deep learning model to obtain multi-scale feature maps corresponding to each MPR image in the MPR image queue. The deep learning model also includes a multi-plane interaction module and a feature fusion and encoding module.

[0034] In some possible implementations, the attention module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule weights channel features using global average pooling and a multilayer perceptron, while the spatial attention submodule enhances spatial features using convolution and a non-linear activation function. The non-linear activation function can be any of the following: Sigmoid, ReLU, or Swish.

[0035] S105. Use the multi-plane interaction module to perform cross-plane association and fusion of multi-scale feature maps to obtain a multi-plane fused feature map with a two-dimensional structure.

[0036] In some possible implementations, the multi-plane interaction module calculates the similarity matrix between the multi-scale feature maps corresponding to each MPR image in the MPR image queue, and adjusts the attention weight values ​​of the collaborative diagnostic features based on the similarity matrix and according to the preset attention weight enhancement rules.

[0037] S106. The feature fusion and encoding module is used to perform global pooling and fully connected encoding on the feature map fused from the multi-plane model, and output a one-dimensional diagnostic vector for auxiliary diagnosis.

[0038] In some possible implementations, the feature fusion and encoding module includes a global average pooling layer and a fully connected layer. The global average pooling layer compresses the spatial dimension of the feature map fused from the multi-plane model, and the fully connected layer maps the compressed features into a one-dimensional diagnostic vector.

[0039] In some possible implementations, disease classification results or probabilities can be output based on a one-dimensional diagnostic vector. For example, after obtaining a one-dimensional diagnostic vector, a classifier can be connected to output the disease classification result or probability. A common approach is to connect a normalized exponential function (Softmax) layer after the diagnostic vector. This layer transforms the vector values ​​into probability values ​​in the range of 0-1, with a sum of 1. The category with the highest probability is the classification result. Alternatively, a Support Vector Machine (SVM) can be used to determine the classification hyperplane through training, thus identifying the category to which the diagnostic vector belongs. Some SVM methods can also provide probability estimates. Furthermore, algorithms such as decision trees and random forests can be used to perform classification decisions using the data from each dimension of the diagnostic vector as features, outputting the classification result and corresponding probability through voting or calculation rules.

[0040] This embodiment enables the efficient conversion of raw medical image data into diagnostic vectors, achieving automated computer-aided diagnosis. Based on a multi-module collaboration of deep learning models, it can fully extract multi-scale and multi-planar features of medical images, effectively integrate global and local information, improve the ability to identify complex lesions, provide doctors with quantitative diagnostic references, and enhance diagnostic efficiency and accuracy while reducing the subjectivity and error of human interpretation.

[0041] This invention also provides a computer-aided diagnostic system based on medical images, including: an MPR image encoder and a deep learning module. The MPR image encoder acquires raw DICOM data obtained using tomographic scanning technology; constructs three-dimensional volumetric data based on the raw DICOM data; and performs multi-plane reconstruction based on the three-dimensional volumetric data to generate an MPR image queue. The deep learning model sequentially passes the MPR image queue through its own convolutional blocks, attention modules, and feature pyramid network to obtain multi-scale feature maps corresponding to each MPR image in the MPR image queue. The deep learning model also includes: a multi-plane interaction module and a feature fusion and encoding module. The multi-plane interaction module performs cross-plane correlation fusion on the multi-scale feature maps to obtain a two-dimensional multi-plane fused feature map; and the feature fusion and encoding module performs global pooling and fully connected encoding on the multi-plane fused feature map to output a one-dimensional diagnostic vector for auxiliary diagnosis.

[0042] The following is combined Figure 4 A schematic diagram of the deep learning module architecture in a deep learning network.

[0043] The sequence of MPR images is the output of the MPR image encoder. Then, the coronal MPR, sagittal MPR, and transverse MPR images to be summarized in the sequence of MPR images are respectively input into the backbone feature extraction module (ResNet or EfficientNet). In parallel, each corresponds to a set of convolutional blocks 1, 2, 3, and 4 to extract features step by step from shallow to deep layers, capturing low-level texture to high-level semantic information.

[0044] The input to the backbone feature extraction module can be a single-plane MPR image (such as a coronal lung image), which extracts features at different levels (from texture to semantics) from the MPR image and outputs feature maps of different depths (shallow layers contain edge textures, and deep layers contain organ semantics).

[0045] Specifically, network structures such as ResNet can be used to extract features step by step through multiple convolutional layers: shallow convolutions identify lines and edges (e.g., the outline of lung lobes); deep convolutions identify the overall shape of the organ (e.g., the outline of the entire lung). For example, after inputting a coronal lung image, the skeletal feature extraction module first finds the lines of the lung edges (shallow layer), and then combines these lines to identify the overall shape of the left and right lungs (deep layer).

[0046] The attention enhancement module adds a Convolutional Block Attention Module (CBAM) after each intermediate layer (Convolutional Block 2, Convolutional Block 3, and Convolutional Block 4). The CBAM module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule uses global average pooling and a multilayer perceptron to weight channel features, while the spatial attention submodule uses convolution and non-linear activation functions to enhance spatial features. This adaptively emphasizes important feature channels and spatial locations, improving the model's ability to focus on key anatomical structures. In simpler terms, it tells the model "where to look," strengthening key region features. Its input is the output (feature map) of the backbone feature extraction module. The output is a weighted feature map (with key regions highlighted). Specifically, the channel attention submodule determines which feature channels (e.g., the "lung nodule texture" channel) are more important; the spatial attention submodule determines which locations in the feature map (e.g., the upper lobe region of the lung) require more attention. For example, if a lung nodule is present in an image, CBAM will assign higher weights to the "nodule texture channel" and "spatial location of the nodule", allowing the model to focus on analyzing that area and ignore irrelevant structures such as ribs.

[0047] The in-plane multi-scale feature fusion module includes three Feature Pyramid Networks (FPNs). Each FPN module independently processes the features of its respective plane (coronal, sagittal, and transverse), outputting a feature map that considers both the overall picture and the details of that plane. Coronal FPN: Integrates large-scale (e.g., overall lung morphology) and small-scale (e.g., nodular spiculation) features of the coronal plane; Sagittal FPN: Integrates large-scale (e.g., the positional relationship between the heart and lungs) and small-scale (e.g., interstitial abnormalities) features in the sagittal plane; Cross-sectional FPN: Integrates large-scale (e.g., layered thoracic structures) and small-scale (e.g., nodular lobulation) features of cross-sections.

[0048] The three feature maps output by FPN are the outputs of the in-plane multi-scale feature fusion module, one for each plane, each integrating information from different scales within that plane. For example, when analyzing lung CT scans, the in-plane multi-scale feature fusion module simultaneously focuses on the "morphology of the entire lung" (large scale) and "small nodules of 2mm in the lung" (small scale), avoiding missed diagnoses due to nodules being too small.

[0049] The multi-plane interaction module connects features from three planes to form a three-dimensional understanding. The inputs to the multi-plane interaction module include FPN feature maps from the coronal plane, sagittal plane, and transverse plane. The output includes cross-plane fused feature maps (such as the correlation features of "coronal nodule spiculation + sagittal nodule location"). Specifically, the multi-plane interaction module can use attention interaction to strengthen the malignancy correlation weight between related features from the three planes, such as when "spiculation" in the coronal plane and "lobulation" in the transverse plane appear simultaneously. For example, looking at the transverse plane alone might suggest a benign nodule, but combining the spiculation features from the coronal plane, the multi-plane interaction module will determine that the probability of malignancy is higher when "spiculation + lobulation" appear simultaneously, thus avoiding misjudgment from a single plane.

[0050] The feature fusion and encoding module compresses cross-plane features into a diagnostic vector. The input to this module can include the feature map output from the multi-plane interaction module. The output is a one-dimensional diagnostic vector (e.g., [0.9, 0.1] represents a 90% probability of malignancy). Specifically, the module can include a global pooling module and a fully connected layer module. The global pooling module extracts global features from each plane (e.g., whether there are abnormalities in the entire lung); the fully connected layer maps the features to diagnostic probabilities. For example, the feature fusion and encoding module integrates features from three planes and outputs a numerical vector. For instance, a value close to 1 indicates a high probability of malignancy, while a value close to 0 indicates a high probability of benignity. This provides doctors with quantitative diagnostic information.

[0051] Figure 5 This is a schematic diagram of the structure of a computer-aided diagnostic device based on medical images provided in an embodiment of the present invention. The computer-aided diagnostic device 500 based on medical images includes: a memory 501 and a processor 502. The processor 502 is configured to execute computer-executable instructions. The memory 501 is used to store one or more computer-executable instructions. When the computer-executable instructions are executed by the processor 502, they implement each step of any of the computer-aided diagnostic methods based on medical images described in the preceding method embodiments.

[0052] This invention also provides a computer storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements any of the embodiments in the preceding embodiments of the computer-aided diagnosis method based on medical images.

[0053] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the embodiments described in the preceding embodiments of the computer-aided diagnosis method based on medical images.

[0054] This invention also provides a chip system for use in an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to execute any of the methods in the preceding embodiments of the computer-aided diagnosis method based on medical images.

[0055] It is understood that the beneficial effects that the above-mentioned computer-aided diagnostic system, computer-aided diagnostic device, computer storage medium, computer program product, and chip system based on medical images can achieve can be referred to the beneficial effects described in the method embodiments, and will not be repeated here.

[0056] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0057] The above descriptions are optional embodiments provided by this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the technical scope disclosed in this application should be included within the protection scope of this application.

Claims

1. A computer-aided diagnostic method based on medical images, characterized in that, Includes the following steps: Acquire raw DICOM data based on tomographic scanning technology; construct three-dimensional volume data based on the raw DICOM data; Multi-plane reconstruction (MPR) is performed based on the three-dimensional volume data to generate an MPR image queue; the MPR image queue is input into a deep learning model, and the deep learning model sequentially passes through convolutional blocks, attention modules, and feature pyramid networks to obtain multi-scale feature maps corresponding to each MPR image in the MPR image queue; The deep learning model also includes: a multi-plane interaction module and a feature fusion and encoding module; The multi-plane interaction module is used to perform cross-plane correlation fusion on the multi-scale feature map to obtain a multi-plane fused feature map with a two-dimensional structure; the feature fusion and encoding module is used to perform global pooling and fully connected encoding on the multi-plane fused feature map to output a one-dimensional diagnostic vector for auxiliary diagnosis.

2. The computer-aided diagnostic method based on medical images according to claim 1, characterized in that, The MPR image queue includes at least: coronal MPR images, sagittal MPR images, and transverse MPR images; the process of generating the MPR image queue based on the three-dimensional volumetric data through multi-plane reconstruction (MPR) includes: Based on the three-dimensional volume data, slices are taken along the coronal axis, sagittal axis, and transverse axis to generate a two-dimensional image sequence in an orthogonal plane. The slice thickness and spacing are configurable.

3. The computer-aided diagnostic method based on medical images according to claim 1, characterized in that, The attention module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule achieves channel feature weighting through global average pooling and a multilayer perceptron, while the spatial attention submodule enhances spatial features through convolution and a nonlinear activation function.

4. The computer-aided diagnostic method based on medical images according to claim 1, characterized in that, The feature pyramid network employs a top-down multi-scale feature fusion strategy, which upsamples and adds element-wise the feature maps at different levels output by the convolutional blocks to generate multi-scale feature maps that contain both global structure and local details.

5. The computer-aided diagnostic method based on medical images according to claim 1, characterized in that, The multi-plane interaction module calculates the similarity matrix between the multi-scale feature maps corresponding to each MPR image in the MPR image queue, and adjusts the attention weight values ​​of the collaborative diagnostic features based on the similarity matrix according to the preset attention weight enhancement rules.

6. The computer-aided diagnostic method based on medical images according to any one of claims 1 to 5, characterized in that, The feature fusion and encoding module includes a global average pooling layer and a fully connected layer. The global average pooling layer compresses the spatial dimension of the feature map fused from the multi-plane, and the fully connected layer maps the compressed features into a one-dimensional diagnostic vector.

7. A computer-aided diagnostic system based on medical images, characterized in that, include: MPR image encoder and deep learning module The MPR image encoder is used to acquire raw DICOM data obtained based on tomographic scanning technology; Three-dimensional volume data is constructed based on the original DICOM data, and multi-plane reconstruction (MPR) is performed based on the three-dimensional volume data to generate an MPR image queue. The deep learning model is used to sequentially pass the MPR image queue through its own convolutional block, attention module, and feature pyramid network to obtain a multi-scale feature map corresponding to each MPR image in the MPR image queue. The deep learning model further includes a multi-plane interaction module and a feature fusion and encoding module, used to perform cross-plane correlation fusion on the multi-scale feature map using the multi-plane interaction module to obtain a two-dimensional multi-plane fused feature map; and to perform global pooling and fully connected encoding on the multi-plane fused feature map using the feature fusion and encoding module to output a one-dimensional diagnostic vector for auxiliary diagnosis.

8. A computer-aided diagnostic device based on medical images, characterized in that, The device includes a processor and a memory, the processor being configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the various steps of the computer-aided diagnosis method based on medical images as described in any one of claims 1-6.

9. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the computer-aided diagnostic method based on medical images as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the computer-aided diagnostic method based on medical images as described in any one of claims 1-6.

Images