Medical imaging lesion detection methods, devices, and procedures
By combining lesion detection models from CT and PET images, and utilizing cross-modal attention modules and adaptive fusion decision networks, the problem of insufficient utilization of multimodal information is solved, improving the accuracy and boundary accuracy of lesion detection and adapting to the detection needs under complex imaging conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING INST OF TECH
- Filing Date
- 2026-01-29
- Publication Date
- 2026-06-02
AI Technical Summary
Existing medical imaging lesion detection methods suffer from insufficient utilization of multimodal information, computational redundancy, error accumulation, and difficulty in meeting real-time clinical needs. Furthermore, they are insufficient in recognizing small-sized, low-contrast lesions, and the stability and reliability of the models under complex imaging conditions need to be improved.
A lesion detection model using CT and PET images is developed. A preprocessing network generates a fusion feature map, which is then combined with the YOLOv5 feature extraction network and cross-modal attention module. Adaptive weighting of structural and functional modal feature maps is used, along with boundary fineness scoring and an adaptive fusion decision network, to improve the accuracy of lesion detection.
It improves the ability to identify small lesions and low-contrast lesions, achieves a balanced optimization of detection confidence and boundary quality, maintains high-quality feature expression and detection accuracy under complex imaging conditions, and reduces the misdiagnosis rate.
Smart Images
Figure CN122134630A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical image processing and computer-aided diagnosis technology, specifically relating to a method, device, and program product for detecting lesions in medical images. Background Technology
[0002] With the rapid development of medical imaging technology, multimodal images such as CT, MRI, and PET have become important bases for clinical diagnosis and treatment planning. However, manual analysis of massive amounts of image data suffers from inefficiency, strong subjectivity, and fatigue, prompting automated analysis technology based on deep learning to become a research hotspot. Current mainstream methods generally adopt an architecture that separates detection and segmentation. First, the lesion region is located through a target detection network, and then a segmentation network is used for fine processing. This serial process leads to computational redundancy, error accumulation, and difficulty in meeting the real-time requirements of clinical practice. At the same time, existing methods utilize multimodal information only superficially, failing to fully explore the complementarity between structural and functional modalities at the feature level, resulting in insufficient ability to identify small-sized, low-contrast lesions.
[0003] On the other hand, the core objective of general object detection models is to achieve high recall and accurate localization. However, the segmentation results they directly output often have coarse boundaries and lost details, which cannot meet the stringent requirements for boundary accuracy in scenarios such as quantitative lesion measurement and surgical planning. In addition, the morphology, contrast, and surrounding environment of lesions in medical images are complex and variable, while existing models mostly use fixed decision thresholds and lack a mechanism to adaptively adjust the weights of detection confidence and segmentation accuracy based on the specific characteristics of the lesion. Therefore, their stability and reliability under complex imaging conditions need to be improved. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a method for detecting lesions in medical imaging, which solves the problem of insufficient utilization of modal information in multimodal medical imaging, thereby improving detection accuracy.
[0005] The present invention achieves the above-mentioned technical objectives through the following technical means.
[0006] A medical imaging lesion detection method, targeting CT and PET images, employs the following lesion detection model for lesion detection, including:
[0007] Preprocessing network:
[0008] The CT images are filtered and normalized, and then the process is divided into two parallel operations:
[0009] A gradient map is generated by calculating the Sobel gradient;
[0010] After grayscale conversion, region maps are generated through pixel similarity clustering;
[0011] The gradient map and the region map are fused, and then a semantic mask is generated by binarization segmentation and isolated pixel filtering.
[0012] For PET images, PET feature maps are generated through quantitative calibration calculations using SUVs.
[0013] The original CT image, PET feature map, and semantic mask are concatenated along the channel dimension to obtain the fused feature map. The output is fed into the feature extraction network;
[0014] Feature extraction network: YOLOv5 backbone network and neck network are used to fuse feature maps. Perform feature extraction;
[0015] Feature detection network: The YOLOv5 detection head is used to detect the feature maps extracted by the feature extraction network.
[0016] Furthermore, in the feature extraction network, cross-modal attention modules are added after the first three CSP modules:
[0017] For the feature map output by the CSP module First, two 1×1 convolutions are used to extract the structural modality feature maps. and functional modal feature map ;
[0018] Structural modal feature map Generate query matrix through 1×1 convolution. Functional modal feature map The key matrix is generated by two 1×1 convolution processes respectively. Sum matrix ;
[0019] Query Matrix AND key matrix Multiply by the transpose and divide by the scaling factor. After Softmax normalization, the sum-value matrix Multiplication yields the enhanced fused feature map:
[0020]
[0021] The enhanced fused feature map is residually connected to the structural modality feature map to obtain the output of the cross-modal attention module.
[0022]
[0023] In the formula This is the output of the cross-modal attention module.
[0024] Furthermore, the feature detection network also includes a segmentation head:
[0025] The feature map extracted by the feature extraction network is first upsampled and then concatenated with the feature map output by the first CSP module in the feature extraction network. After concatenation, the mask image is obtained by performing a 3×3 convolution and a mask prediction head. ;
[0026] Based on mask image Calculate its fusion feature map Structural similarity between Calculate its fusion feature map Gradient matching degree between Calculate the mask image Self-boundary gradient clarity ;
[0027] Will , , Weighted summation yields the boundary fineness score. ;
[0028] Finally, the confidence level obtained based on the detection head is considered. Compared with the above boundary fineness score The validity of the detection results from the detection head is determined.
[0029] Furthermore, the lesion detection model incorporates a decision fusion network:
[0030] The confidence level With the boundary fineness score Perform a weighted summation to obtain the overall score. :
[0031]
[0032] In the formula, As weight;
[0033] Comprehensive score based on detection threshold Make a judgment, where:
[0034] If the result exceeds the detection threshold, the detection result output by the detection head is considered valid.
[0035] If the result is less than or equal to the detection threshold, the detection result output by the detection head is considered invalid.
[0036] Furthermore, the weights Obtained through the following methods:
[0037] Based on the mask, the corresponding local images are extracted from the original CT and PET images;
[0038] Texture entropy and gradient variance are calculated based on local CT images.
[0039] Calculate noise level based on local PET images;
[0040] The texture entropy, gradient variance, and noise level are weighted and summed, and the result is then activated by the Sigmoid activation function to obtain the final result. .
[0041] Furthermore, the aforementioned The calculation is as follows:
[0042]
[0043] In the formula, and These are mask images. The mean and variance, and Feature maps are fused separately Mean and variance For mask image With fusion feature map Covariance between and This is a constant used to maintain computational stability.
[0044] Furthermore, the aforementioned The calculation is as follows:
[0045]
[0046] In the formula, The gradient vector of the masked image edge. To fuse the edge gradient vectors at corresponding locations in the feature map, the subscript... For the pixel variables at the boundary positions of the mask image, This represents the total number of pixels on the boundary of the masked image.
[0047] Furthermore, the aforementioned The calculation is as follows:
[0048]
[0049] In the formula, For mask image The total length of the boundary curve, Represents a mask image Boundary curve, Boundary curve Pixel rate of change at that point Boundary curve The arc length infinitesimal element on the surface.
[0050] A computer device, including a memory and a processor;
[0051] The memory is used to store computer programs;
[0052] The processor is used to execute the computer program and, in executing the computer program, implement the above-described medical imaging lesion detection method.
[0053] A computer program product includes a computer program that, when executed by a processor, implements the above-described method for detecting lesions in medical images.
[0054] The beneficial effects of this invention are as follows:
[0055] (1) This invention provides a method for detecting lesions in medical images. By using structural prior constraints (i.e., the fusion of CT, PET, and semantic mask) and cross-modal attention mechanism, it strengthens the association between structural and functional modalities and improves the ability to identify small lesions and low-contrast lesions.
[0056] (2) This invention achieves a balance between detection confidence and boundary quality by combining boundary fineness scoring with adaptive fusion.
[0057] (3) The cross-modal attention module designed in this invention enables the model to dynamically select more reliable information sources by adaptively weighting the structural and functional modalities. For example, CT has strong boundary information, while PET has more prominent metabolic features. The cross-modal attention mechanism can automatically find the optimal fusion strategy between the two, thus obtaining high-quality feature expression even when the tumor boundary is blurred, the tissue structure is complex, or the background noise is strong. This invention learns the importance of each modality in different spatial locations through the cross-modal attention module, enabling the network to automatically adjust the fusion strategy when facing complex modal combinations, thereby ensuring that the information of small lesions and low-contrast areas is not overwhelmed by high-energy modalities (such as PET). In other words, this method achieves true multimodal collaborative learning, rather than simple splicing or weighted superposition.
[0058] (4) This invention comprehensively utilizes structural similarity Gradient matching degree Boundary gradient clarity Among them, SSIM serves as a macro-structural consistency indicator, and GM serves as a local detail matching indicator. As a boundary sharpness metric, it provides a comprehensive rating of the quality of boundary predictions. This score can be used for both loss function optimization during model training and quality assessment during inference.
[0059] (5) To overcome the impact of variations in lesion complexity (such as tissue heterogeneity, imaging noise, blurred tumor boundaries, and organ overlap) on detection confidence, this invention designs an adaptive scoring fusion strategy based on lesion complexity parameters. First, lesion complexity parameters are estimated based on features such as texture variations, local variance, and noise distribution. Then, adaptive weights are obtained using an activation function. Ultimately, through this strategy, the detection model can automatically balance detection confidence and boundary prediction quality under different image types and lesion structures, significantly improving the detection capability and boundary precision of small lesions. Attached Figure Description
[0060] Figure 1 This is a diagram showing the overall network structure of the lesion detection model of the present invention;
[0061] Figure 2 This is a structural diagram of the specially extracted network part in the lesion detection model of the present invention;
[0062] Figure 3 This is a network structure diagram of the cross-modal attention module in the lesion detection model of the present invention;
[0063] Figure 4 This is a structural diagram of the special diagnosis detection network part in the lesion detection model of the present invention;
[0064] Figure 5 For corresponding Figure 4 A detailed structural diagram of the detection head and segmentation head for any scale feature;
[0065] Figure 6 This is a structural diagram of the decision fusion network part in the lesion detection model of the present invention;
[0066] Figure 7 This is a training performance curve of the lesion detection model of the present invention;
[0067] Figure 8 This is the F1-confidence curve of the lesion detection model of the present invention;
[0068] Figure 9 This is a comparison of test samples between the present invention and the existing network model control group. Detailed Implementation
[0069] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.
[0070] I. Detection Methods
[0071] like Figure 1As shown, this embodiment uses a lesion detection neural network model (hereinafter referred to as the lesion detection model) built based on YOLOv5 to perform medical image lesion detection, wherein the original input data used for detection are CT images and PET images. The lesion detection model includes the following parts:
[0072] 1. Preprocessing network
[0073] The acquired CT and PET images are preprocessed as follows to obtain the fused feature map:
[0074] (1) For CT images, Gaussian filtering and normalization are performed first; then the following two parallel operations are performed:
[0075] A gradient map is generated by calculating the Sobel gradient (i.e., the value of each pixel is its gradient magnitude).
[0076] After being converted to grayscale, region maps are generated through pixel similarity clustering;
[0077] The gradient map and region map are fused together, and then a semantic mask is generated by binarization segmentation and isolated pixel filtering. The semantic mask contains labels for "structural edge", "background region" and "functionally relevant region".
[0078] (2) For PET images, PET feature maps are generated through quantitative calibration calculations using SUVs;
[0079] (3) The original CT image, PET feature image, and semantic mask are concatenated along the channel dimension to obtain the fused feature image. And input it into the feature extraction network.
[0080] 2. Feature Extraction Network
[0081] like Figure 2 As shown, the feature extraction network specifically adopts the YOLOv5 backbone network and neck network, with cross-modal attention modules added after the first three CSP modules of the original YOLOv5 backbone network.
[0082] like Figure 3 The cross-modal attention module shown:
[0083] 1) Feature maps output by the CSP module First, two 1×1 convolutions are used to extract the structural modality feature maps. and functional modal feature map ;
[0084] 2) Structural modal characteristic diagram Generate query matrix through 1×1 convolution. Functional modal feature map The key matrix is generated by two 1×1 convolution processes respectively. Sum matrix ;
[0085] 3) Query matrix AND key matrix Multiply by the transpose and divide by the scaling factor. After Softmax normalization, the sum-value matrix Multiplication yields the enhanced fused feature map:
[0086]
[0087] In the formula, Key matrix Feature dimensions, Used to scale the dot product result to prevent gradient vanishing.
[0088] 4) The enhanced fused feature map is residually connected to the structural modality feature map to obtain the output of the cross-modal attention module:
[0089]
[0090] In the formula This is the output of the cross-modal attention module.
[0091] 3. Feature Detection Network
[0092] like Figure 4 and Figure 5 As shown, in addition to directly using the three detection heads from the original YOLOv5, three new segmentation heads are added to correspond to the three different scales of feature maps extracted. , , Conduct testing.
[0093] 3.1 For any scale of feature map extracted by the feature extraction network, the target detection result is obtained by sequentially performing 3×3 convolution and 1×1 convolution within the detection head, including bounding box, category, and confidence score. .
[0094] 3.2 For any scale of feature map extracted by the feature extraction network, the scale is first upsampled to be the same as the output feature map of the first CSP module. Then, it is concatenated with the feature map output by the first CSP module in the feature extraction network according to the channel dimension, thereby realizing cross-layer connection of shallow features. After concatenation, a binary segmentation mask (i.e., mask image) is obtained by processing with a 3×3 convolution and a mask prediction head. ).
[0095] The following three evaluation metrics are calculated based on the masked image:
[0096] 1) SSIM
[0097] Calculate the mask image With fusion feature map SSIM (obtained from the preprocessed network) is used to measure the structural similarity between the two:
[0098]
[0099] In the formula, and These are mask images. The mean and variance, and Feature maps are fused separately Mean and variance For mask image With fusion feature map Covariance between and This is a constant used to maintain computational stability.
[0100] 2) GM
[0101] Calculate the mask image With fusion feature map Gradient matching degree between them:
[0102]
[0103] In the formula, The gradient vector of the masked image edge. To fuse the edge gradient vectors at corresponding locations in the feature map, the subscript... For the pixel variables at the boundary positions of the mask image, This represents the total number of pixels on the boundary of the masked image.
[0104] 3)
[0105] Calculate the mask image Self-boundary gradient clarity:
[0106]
[0107] In the formula, For mask image The total length of the boundary curve, Represents a mask image Boundary curve, Boundary curve Pixel The rate of change at a point (i.e., the gradient length). Boundary curve The arc length infinitesimal element on the surface.
[0108] Finally, the three evaluation indicators are weighted and summed to obtain the boundary fineness score. :
[0109]
[0110] In the formula, , , These are the weighting coefficients.
[0111] The output results of each detection head and the output results of each segmentation head at the corresponding scale are further fused by the decision fusion network.
[0112] 4. Decision-making fusion network
[0113] like Figure 6 As shown, firstly, based on a mask, corresponding local images are extracted from the original CT and PET images; then, texture entropy and gradient variance are calculated based on the CT local images, and noise level is calculated based on the PET local images; finally, the three (texture entropy, gradient variance, and noise level) are weighted and summed, and the result is activated by the Sigmoid activation function to obtain an adaptive weighted sum. .
[0114] Then, based on the above adaptive weights Detection confidence Boundary fineness score We perform a weighted summation to obtain the overall score. :
[0115]
[0116] Finally, the overall score is calculated based on the preset detection threshold. Make a judgment, where when If the result exceeds the detection threshold, the detection result output by the detection head is considered valid; otherwise, when... If the result is less than or equal to the detection threshold, the detection result output by the detection head is considered invalid.
[0117] II. Testing and Verification
[0118] Five hundred lung CT / PET multimodal image samples were selected, including:
[0119] Conventional group: Samples with clear imaging and obvious lesion margins (350 cases).
[0120] High complexity group: Simulates extreme clinical conditions, including samples with high background noise, low contrast, or blurred lesion edges (150 cases).
[0121] The above test samples were tested using the detection model of this invention and the detection model that is directly extended from the original YOLOv5 (only preprocessing is added to stitch the multimodal input images).
[0122] like Figure 7 The figure shows the performance curves of the detection model during the training process of this invention. It demonstrates that after 300 training iterations (Epochs), all loss function metrics steadily decreased without overfitting oscillations. The precision, recall, and mean precision (mAP50) all showed significant increasing trends and eventually stabilized above 0.95, fully verifying that the algorithm possesses extremely high diagnostic accuracy, missed diagnosis suppression capability, and convergence stability in complex multimodal environments.
[0123] like Figure 8 As shown, the detection model of this invention achieves an F1 score of 0.95 across all categories at a confidence level of 0.509, demonstrating extremely strong target acquisition capabilities.
[0124] like Figure 9 As shown, column (a) on the left contains the original YOLOv5 detection samples, and column (b) on the right contains the detection samples of this invention. In multiple challenging samples, the control group misidentified "nodule" as "cancer" with a confidence level of only 0.3; this invention identifies the current image complexity as high by calculating texture entropy and noise level, and the decision fusion network dynamically adjusts the weights. The classification was corrected to "nodule," and the confidence levels were increased to 0.6 and 0.9 respectively, which greatly reduced the misdiagnosis rate.
[0125] Other specific test results are shown in Tables 1 and 2 below:
[0126] Table 1: Comparison of lesion segmentation and edge sharpness
[0127]
[0128] Table 2: Adaptive weight distribution under different complexity environments
[0129]
[0130] III. Devices, storage media, and software products
[0131] 1. Based on the same inventive concept as the above-mentioned medical imaging lesion detection method, this application also provides an electronic device, which includes a processor and a memory, wherein computer-readable code is stored in the memory, and when the computer-readable code is executed by the processor, the medical imaging lesion detection method of the present invention is implemented.
[0132] The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium can store the operating system and computer-readable code. The computer-readable code includes program instructions that, when executed, cause the processor to perform a medical imaging lesion detection method. The processor provides computational and control capabilities to support the operation of the entire electronic device. The memory provides an environment for the execution of the computer-readable code in the non-volatile storage medium, which, when executed by the processor, causes the processor to perform the medical imaging lesion detection method.
[0133] It should be understood that a processor can be a central processing unit, other general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, transistor logic devices, discrete hardware components, etc. Among them, a general-purpose processor can be a microprocessor or any conventional processor.
[0134] 2. This application also provides a readable storage medium, which may be an internal storage unit of the electronic device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, smart memory card, or secure digital card equipped on the electronic device.
[0135] 3. This application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the medical imaging lesion detection method of the present invention.
[0136] This invention is not limited to the above-described embodiments. Any obvious improvements, substitutions, or modifications that can be made by those skilled in the art without departing from the essence of this invention are within the scope of protection of this invention.
Claims
1. A method for detecting lesions in medical imaging, characterized in that: For CT and PET images, the following lesion detection models are used for lesion detection, including: Preprocessing network: The CT images are filtered and normalized, and then the process is divided into two parallel operations: A gradient map is generated by calculating the Sobel gradient; After grayscale conversion, region maps are generated through pixel similarity clustering; The gradient map and the region map are fused, and then a semantic mask is generated by binarization segmentation and isolated pixel filtering. For PET images, PET feature maps are generated through quantitative calibration calculations using SUVs. The original CT image, PET feature map, and semantic mask are concatenated along the channel dimension to obtain the fused feature map. The output is fed into the feature extraction network; Feature extraction network: YOLOv5 backbone network and neck network are used to fuse feature maps. Perform feature extraction; Feature detection network: The YOLOv5 detection head is used to detect the feature maps extracted by the feature extraction network.
2. The method for detecting lesions in medical imaging according to claim 1, characterized in that: In the feature extraction network, cross-modal attention modules are added after the first three CSP modules: For the feature map output by the CSP module First, two 1×1 convolutions are used to extract the structural modality feature maps. and functional modal feature map ; Structural modal feature map Generate the query matrix through 1×1 convolution. Functional modal feature map The key matrix is generated by two 1×1 convolution processes respectively. Sum matrix ; Query Matrix AND key matrix Multiply by the transpose and divide by the scaling factor. After Softmax normalization, the sum-value matrix Multiplication yields the enhanced fused feature map: The enhanced fused feature map is residually connected to the structural modality feature map to obtain the output of the cross-modal attention module. In the formula This is the output of the cross-modal attention module.
3. The method for detecting lesions in medical imaging according to claim 1, characterized in that: The feature detection network also includes a segmentation head: The feature map extracted by the feature extraction network is first upsampled and then concatenated with the feature map output by the first CSP module in the feature extraction network. After concatenation, the mask image is obtained by performing a 3×3 convolution and a mask prediction head. ; Based on mask image Calculate its fusion feature map Structural similarity between Calculate its fusion feature map Gradient matching degree between Calculate the mask image Self-boundary gradient clarity ; Will , , Weighted summation yields the boundary fineness score. ; Finally, the confidence level obtained based on the detection head is considered. Compared with the above boundary fineness score The validity of the detection results from the detection head is determined.
4. The method for detecting lesions in medical imaging according to claim 3, characterized in that: The lesion detection model incorporates a decision fusion network. The confidence level With the boundary fineness score Perform a weighted summation to obtain the overall score. : In the formula, As weight; Comprehensive score based on detection threshold Make a judgment, where: If the result exceeds the detection threshold, the detection result output by the detection head is considered valid. If the result is less than or equal to the detection threshold, the detection result output by the detection head is considered invalid.
5. The method for detecting lesions in medical imaging according to claim 4, characterized in that: The weight Obtained through the following methods: Based on the mask, the corresponding local images are extracted from the original CT and PET images; Texture entropy and gradient variance are calculated based on local CT images. Calculate noise level based on local PET images; The texture entropy, gradient variance, and noise level are weighted and summed, and the result is then activated by the Sigmoid activation function to obtain the final result. .
6. The method for detecting lesions in medical imaging according to claim 3, characterized in that: The The calculation is as follows: In the formula, and These are mask images. The mean and variance, and Feature maps are fused separately Mean and variance For mask image With fusion feature map Covariance between and This is a constant used to maintain computational stability.
7. The method for detecting lesions in medical imaging according to claim 3, characterized in that: The The calculation is as follows: In the formula, The gradient vector of the masked image edge. To fuse the edge gradient vectors at corresponding locations in the feature map, the subscript... For the pixel variable at the boundary position of the mask image, This represents the total number of pixels on the boundary of the masked image.
8. The method for detecting lesions in medical imaging according to claim 1, characterized in that: The The calculation is as follows: In the formula, For mask image The total length of the boundary curve, Represents a mask image Boundary curve, Boundary curve Pixel rate of change at that point Boundary curve The arc length infinitesimal element on the surface.
9. A computer device, characterized in that: Including memory and processor; The memory is used to store computer programs; The processor is used to execute the computer program and, in executing the computer program, implement the medical imaging lesion detection method as described in any one of claims 1 to 8.
10. A computer program product, characterized in that: It includes a computer program, which, when executed by a processor, implements the medical imaging lesion detection method as described in any one of claims 1 to 8.