Medical image lesion automatic recognition system based on convolutional neural network

Abstract

The application discloses a medical image lesion automatic identification system based on a convolutional neural network, relates to the field of image recognition, and comprises a multi-modal data adaptation module that receives spectral CT dual-energy, MRI multi-sequence and clinical metadata; a dynamic enhancement convolution module that is based on an improved Mallat algorithm, adjusts wavelet coefficients with an adaptive threshold, adopts a spatial and channel attention cascade structure, generates a fusion weight matrix to enhance lesion features; a lesion shape constraint module that extracts lesion candidate area parameters and generates a calibration feature map; and a cross-scale feature fusion module that constructs four feature scale layers and fuses features through dynamic weights that are jointly adjusted according to information entropy and shape matching degree. The medical image lesion automatic identification system based on the convolutional neural network realizes deep fusion and accurate screening of multi-modal features through a shape constraint and a multi-model integration mechanism, further improves lesion recognition accuracy in a small sample scene, reduces the false positive rate, and enhances decision reliability.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to an automatic medical image lesion recognition system based on convolutional neural networks. Background Technology

[0002] Medical image lesion identification is a core supporting technology for precision medicine. Convolutional neural networks (CNNs) have been widely used in this field due to their feature extraction capabilities. Existing lesion identification methods use wavelet filtering to enhance image clarity and then construct three-dimensional volume data, and use multi-scale feature pyramid networks to extract features. There are three major drawbacks: First, relying on the enhancement processing of a single modality of image cannot integrate the functional parameter information of new equipment such as spectral CT, resulting in insufficient ability to identify occult lesions with similar densities. Second, the feature extraction process does not distinguish the weight difference between the core and edge regions of the lesion, and lacks targeted constraints on the morphological features of the lesions, making it susceptible to interference from background tissues, leading to an increase in false positives. For example, the false positive rate in lung CT screening is still as high as 30%-50%. Third, model training relies on large-scale labeled data, and the generalization performance drops sharply in small sample scenarios such as rare lesions. Moreover, the decision of a single model is easily affected by data distribution bias, resulting in insufficient reliability. In addition, existing systems often separate image enhancement and feature fusion into independent modules, lacking an end-to-end collaborative optimization mechanism, which leads to serious information loss during feature transfer. Summary of the Invention

[0003] The purpose of this invention is to provide an automatic medical image lesion identification system based on convolutional neural networks, in order to solve the problems mentioned in the background art, such as insufficient ability to identify hidden lesions with similar density, failure to distinguish the weight difference between the core and edge regions of the lesion during feature extraction, model training relying on large-scale labeled data, sharp decline in generalization performance in small sample scenarios such as rare lesions, and the susceptibility of single model decision-making to data distribution bias, as well as the lack of a collaborative mechanism, resulting in serious information loss during feature transmission.

[0004] To achieve the above objectives, the present invention provides the following technical solution: an automatic medical image lesion recognition system based on convolutional neural networks, comprising: Multimodal data adaptation module: Receives spectral CT dual-energy data, MRI multi-sequence data and clinical metadata, and constructs a 512×512×64×C-dimensional multimodal tensor matrix through linear interpolation resampling and normalization processing; Dynamically Enhanced Convolutional Module: Based on Improved The algorithm introduces an adaptive threshold parameter to adjust the wavelet coefficients. Multi-scale features are extracted by combining three parallel convolutional branches with different dilation rates; Lesion-background attention module: It adopts a cascaded structure of spatial attention and channel attention, calculates channel dependence through feature cloud dot product attention, and generates a fusion weight matrix to enhance lesion features; Lesion morphology constraint module: Extracts shape parameters, roundness, lobulation index and texture parameters of lesion candidate regions, performs dynamic constraint filtering based on a preset threshold library, generates a calibration feature map of semantic features + morphological features, and works in collaboration with lesion-background attention module and cross-scale feature fusion module. Cross-scale feature fusion module: Constructs 4 feature scale layers, and fuses features through dynamic weights jointly adjusted by information entropy and morphological matching degree, preserving lesion details and global context information; Multi-model ensemble decision module: based on ResNet50, U-Net constructs three base models and uses a confidence-weighted voting + threshold calibration strategy to output recognition results, forming a collaborative closed loop with the self-supervised optimization module; Self-supervised optimization module: Optimizes the model through unlabeled data pre-training, contrastive learning fine-tuning and hard sample mining, and dynamically updates the model weights and morphological constraint threshold library by combining the decision results of multi-model integration. The modules are connected in series to form an end-to-end recognition framework.

[0005] In this embodiment, the multimodal data adaptation module is used to receive multi-source medical image data, wherein the dual-energy imaging data specifications of spectral CT are: high energy 140kVp and low energy 80kVp. MRI multi-sequence data includes T1-weighted and T2-weighted images from MRI. The corresponding clinical metadata includes initial screening information on lesion location, patient physiological parameters, and past medical history tags; Data standardization is achieved by resampling image data to a uniform resolution of 512×512×64 using linear interpolation, and normalizing clinical metadata to convert it into feature vectors. A multimodal tensor matrix is ​​constructed, and the standardized image data and clinical metadata are concatenated through feature concatenation to generate an input tensor with a dimension of 512×512×64×C. Where C represents the number of modal channels, with spectral CT data occupying 2 channels, MRI data occupying 2 channels, and clinical metadata occupying 1 channel.

[0006] In this embodiment, the adaptive threshold adjustment mechanism in the dynamically enhanced convolution module enhances the image using the following formula:

[0007] in, The adaptive threshold parameter is dynamically generated from the local contrast of the input tensor. When the local contrast is lower than the preset threshold of 0.15, Automatic enlargement is used to enhance edge features; three parallel dilated convolution branches are set with dilation rates of 1, 3, and 5, respectively, to perform multi-scale preliminary extraction on the enhanced feature map, followed by group normalization. It replaces batch normalization processing, reducing the impact of fluctuations during training with small batches of data.

[0008] In this embodiment, the spatial attention submodule performs global average pooling and global max pooling on the feature map output by the dynamically enhanced convolutional module, concatenates the results, and generates a spatial attention map through a 1×1 convolution. The weights of the lesion region are calculated using the following formula:

[0009] in, for Activation function This is a 1×1 convolution operation. Input feature map; Channel attention submodule: Based on a nonlocal shape analysis mechanism, it constructs a voxel-level feature cloud from the feature map. ,in, For the number of voxels, Number of channels; Calculate inter-channel dependencies using dot product attention:

[0010] The spatial attention map and the channel attention map are multiplied element by element to generate a fusion weight matrix, which is used to weight and enhance the original feature map, highlighting the features of the core lesion region.

[0011] In this embodiment, the lesion morphology constraint module serves as the core module for feature screening and optimization. Based on prior knowledge of clinical lesion morphology, such as the fact that lung cancer lesions are mostly lobulated and brain metastases are mostly round, a constraint mechanism is constructed: Morphological feature extraction: Morphological analysis is performed on the attention-weighted feature map to extract the geometric parameters of the lesion candidate region, including: Shape parameters: Roundness: , For the area, Perimeter; Fragmentation Index ; Texture parameters: contrast and entropy values ​​of the gray-level co-occurrence matrix (GLCM), reflecting the density uniformity within the lesion; Dynamic constraint filtering: A morphological constraint threshold library is constructed, with preset threshold ranges for different lesion types, such as a roundness of 0.3-0.7 and a lobulation index ≥1.2 for lung cancer; and a roundness of 0.7-0.9 and a lobulation index ≤1.1 for brain metastases, to filter candidate regions. When the morphological parameters of the candidate region fall within the threshold range of the corresponding lesion type, its feature weight is retained and enhanced, and the weight coefficient is increased to 1.2-1.5; When the morphological parameters exceed the threshold range and the deviation is ≥30%, it is marked as a suspected interference region, and its feature weight is reduced, with the weight coefficient reduced to 0.5-0.8; Feature calibration output: The filtered feature map is concatenated with the morphological parameter feature vector to generate a calibration feature map with a 20% increase in dimension, which is then passed to the cross-scale feature fusion module to achieve dual constraints of semantic features and morphological features.

[0012] In this embodiment, the cross-scale feature fusion module employs an improved Feature Pyramid Network (FPN), which works in deep collaboration with the calibration feature map output by the lesion morphology constraint module. The four feature scale layers have resolutions of 1 / 2, 1 / 4, 1 / 8, and 1 / 16. Low-level feature enhancement: At half the resolution of the low-dimensional feature layer, residual connections and morphological constraint weights are introduced to preserve the edge details and morphological features of small lesions. High-level feature optimization: At a high-dimensional feature layer with a resolution of 1 / 16, 3D contextual information is extracted through 3D convolution, and semantic-morphological fusion lesion volume features are constructed by combining morphological parameters. Feature fusion mechanism: Features at different scales are fused by using dynamic weights through parallel connection layers. The weight parameters are jointly adjusted by the information entropy and morphological matching degree of each layer's features. The information entropy weight accounts for 0.6, and the morphological matching degree weight accounts for 0.4. The higher the information entropy and the better the morphological matching degree, the larger the feature layer weight coefficient, with a value range of 0.1-0.4. Output processing: Global average pooling is performed on the fused features, and the class probability vector is output through a fully connected layer. The preliminary identification results are obtained by non-maximum suppression (NMS) filtering, which includes lesion location, class, and confidence level.

[0013] In this embodiment, the multi-model integrated decision module is used to improve the reliability of the recognition results, forming a collaborative closed loop with the cross-scale feature fusion module and the self-supervised optimization module: Base model construction: Three independent base models are built based on different backbone networks, including: Using ResNet50 as the backbone network, focusing on local feature extraction; :by For the backbone network, the focus is on global context modeling; Using U-Net as the backbone network, focusing on lesion boundary segmentation; Model training collaboration: The pre-trained weights of the self-supervised optimization module are used to initialize three base models, and complementary data augmentation strategies are adopted in the fine-tuning stage; Using rotation and flipping, Use partial occlusion, Gamma correction is used to avoid model overfitting; Integrated decision-making mechanism: Employing a strategy of confidence-weighted voting + threshold calibration. ① Receive the preliminary identification results from 3 base models and extract the lesion category probability and confidence level output by each model; ② Calculate the weighted voting score: ,in, For the first The class probabilities of each model These are the model weights, dynamically allocated based on the accuracy of the validation set. Higher accuracy results in larger weights, with a total of 1. ③ Set dynamic decision threshold: When the weighted voting score is ≥0.85, the recognition result is directly output; when the score is between 0.6 and 0.85, the morphological parameters of the lesion morphology constraint module are called for secondary verification. If the morphological matching degree is ≥0.7, the result is confirmed; otherwise, it is marked as a suspected lesion; when the score is <0.6, no definite lesion is output. Results Feedback: The integrated decision results are fed back to the self-supervised optimization module as a basis for hard sample mining and to continuously optimize the weight allocation of the base model.

[0014] In this embodiment, the self-supervised optimization module is used to solve the problem of small sample training and works in conjunction with the multi-model ensemble decision module for optimization. Pre-training phase: Self-supervised training is performed using unlabeled public medical image datasets, such as LUNA16 and BraTS2021. Data augmentation strategies such as image rotation, flipping, and local occlusion are adopted to train the feature extraction backbone network of the three base models. Fine-tuning phase: Combine the pre-trained model with a small number of labeled target lesion data (≥30 cases), and introduce a contrastive learning loss function:

[0015] in, For cosine similarity, For temperature parameters, the value is 0.07. These are feature vectors of similar lesions; False positive suppression: Construct a false positive sample library containing easily confused regions such as vascular calcification and inflammatory hyperplasia, combine samples of suspected lesions labeled by a multi-model integrated decision module, and optimize the classifier through a hard sample mining strategy to reduce the false positive rate; Model weight updates: Based on the correction results of clinical feedback and the correct labels after physician review, the integrated weights of the three base models are adjusted regularly to ensure that the system adapts to changes in the distribution of clinical data.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This automatic medical image lesion recognition system based on convolutional neural networks achieves deep fusion and accurate screening of multimodal features through morphological constraints and multi-model integration mechanisms, further improving the lesion recognition accuracy in small sample scenarios, reducing the false positive rate, and enhancing the reliability of decision-making.

[0017] Furthermore, by deeply integrating prior knowledge of clinical morphology through the lesion morphology constraint module, the geometric and textural features of lesions are incorporated into the feature system, forming a dual recognition dimension of functional parameters + morphological features with multimodal data. This solves the problem of insufficient recognition capability of traditional single-modal imaging for occult lesions with similar density and blurred boundaries. The dynamic threshold library of the morphology constraint module supports custom expansion, and parameters can be flexibly adjusted according to the morphological features of different lesion types to adapt to the diagnostic needs of lesions in multiple locations and of multiple types, thus solving the shortcomings of existing technologies that are only optimized for a single lesion type and have poor generalization.

[0018] Furthermore, the lesion-background attention module precisely focuses on the core area of ​​the lesion through spatial and channel-based weighting, suppressing background interference such as vascular calcification, inflammatory proliferation, and artifacts; the multi-model integrated decision module further utilizes ResNet50 local features, The complementary modeling of three base models—global context, U-Net boundary segmentation, and confidence-weighted voting combined with morphological secondary verification—double-filters false positive signals at both the feature extraction and decision output levels. Simultaneously, the labeling mechanism for suspected lesions reduces the potential missed diagnosis rate, achieving a balance between low false positives and low missed diagnoses. This addresses the contradiction in existing technologies where either the false positive rate is too high or the risk of missed diagnoses increases. The dynamic weight allocation mechanism of the multi-model ensemble decision-making system adaptively adjusts the base model weights based on the distribution characteristics of image data from different hospitals and devices, ensuring stable recognition performance under various hardware conditions and reducing the risk of misjudgment caused by data distribution bias.

[0019] Furthermore, addressing the clinical challenge of scarce labeled data for rare lesions and special populations, the self-supervised optimization module utilizes a feature extraction network pre-trained on a large-scale unlabeled public dataset, combined with a contrastive learning loss function to enhance the aggregation of features of similar lesions and the discriminative power of dissimilar features. The multi-model ensemble decision further employs a complementary data augmentation strategy, with different base models using differentiated data augmentation methods to expand the feature diversity of effective training samples and reduce the risk of overfitting from small-sample training.

[0020] Furthermore, multimodal adaptation, dynamic enhancement, attention weighting, morphological constraints, feature fusion, integrated decision-making, and self-supervised optimization are integrated into an end-to-end collaborative framework. Through engineering techniques such as model parallel computing and feature map dimension optimization, the recognition time for a single CT image is slightly increased, while the processing speed is improved, resulting in better efficiency compared to traditional modular processing solutions. The system supports deployment in multiple hardware environments, enabling rapid batch image processing on high-performance GPU servers in tertiary hospitals. It can also be adapted to edge computing devices in primary hospitals through model quantization. After quantization, the model size is compressed, and the recognition time is short, meeting the deployment needs of medical institutions at different levels and solving the problem that existing high-end medical image recognition systems rely on high-performance hardware and are difficult to promote to primary care settings.

[0021] The system not only outputs core information such as lesion location, category, and confidence level, but also provides morphological parameter analysis results simultaneously. This provides doctors with auxiliary references for differentiating between benign and malignant lesions and for pathological classification. Compared with existing identification systems that only output the presence or absence of lesions, it is more in line with the clinical diagnostic process and helps doctors improve diagnostic efficiency and accuracy. The self-supervised optimization module forms a closed loop with clinical feedback. It regularly updates the model weights and morphological constraint threshold library based on the label data corrected by doctors. With the accumulation of clinical application data, the system's recognition accuracy continues to iterate and improve, achieving continuous optimization with longer use and better performance, empowering the sustainable improvement of the entire precision medicine process. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the automatic medical image lesion recognition system based on convolutional neural networks according to the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] This invention provides an automatic medical image lesion identification system based on convolutional neural networks. Its core is to receive dual-energy spectral CT, multi-sequence MRI, and clinical metadata, and construct a 512×512×64×C dimension multimodal tensor matrix through linear interpolation, resampling, and normalization. Based on improved... The algorithm uses adaptive thresholding to adjust wavelet coefficients and combines three parallel convolutional branches with different dilation rates to extract multi-scale features. It employs a cascaded spatial and channel attention structure, calculating channel dependence through feature cloud dot product attention to generate a fusion weight matrix that enhances lesion features. It extracts shape and texture parameters of lesion candidate regions, dynamically filters them using a preset threshold library, and generates semantic + morphological calibration feature maps, which collaborate with attention and cross-scale fusion modules. It constructs four feature scale layers, fusing features through dynamic weights jointly adjusted by information entropy and morphological matching degree, preserving lesion details and global contextual information. Based on ResNet50, U-Net constructs three base models, and the output results of confidence-weighted voting + threshold calibration form a collaborative closed loop with the self-supervised optimization module. The model is optimized through unlabeled pre-training, contrastive learning fine-tuning and hard sample mining, and the model weights and morphological constraint threshold library are dynamically updated in combination with the integrated decision results. The modules are sequentially connected to form an end-to-end recognition framework. By adding morphological constraints and multi-model integration mechanisms, deep fusion and accurate screening of multimodal features are achieved, which further improves the lesion recognition accuracy in small sample scenarios, reduces the false positive rate, and enhances the reliability of decision-making.

[0025] Example 1: To better understand the above technical solution, the following will provide a detailed description of the technical solution in conjunction with the accompanying drawings and specific implementation methods. (Refer to...) Figure 1 As shown, the Figure 1 This is a schematic diagram of an automatic medical image lesion recognition system based on convolutional neural networks. The system includes the following modules: Multimodal data adaptation module: Receives spectral CT dual-energy data, MRI multi-sequence data and clinical metadata, and constructs a 512×512×64×C-dimensional multimodal tensor matrix through linear interpolation resampling and normalization processing; Specifically, the multimodal data adaptation module is used to receive multi-source medical image data, among which the dual-energy imaging data specifications of spectral CT are: high energy 140kVp and low energy 80kVp. MRI multi-sequence data includes T1-weighted and T2-weighted images from MRI. The corresponding clinical metadata includes initial screening information on lesion location, patient physiological parameters, and past medical history tags; Data standardization is achieved by resampling image data to a uniform resolution of 512×512×64 using linear interpolation, and normalizing clinical metadata to convert it into feature vectors. A multimodal tensor matrix is ​​constructed, and the standardized image data and clinical metadata are concatenated through feature concatenation to generate an input tensor with a dimension of 512×512×64×C. Where C represents the number of modal channels, with spectral CT data occupying 2 channels, MRI data occupying 2 channels, and clinical metadata occupying 1 channel; It should be further explained that when receiving multi-source data: Spectral CT dual-energy imaging data comes from spectral CT equipment, uses dual-band scanning data, is three-dimensional voxel data, and retains the original physical characteristics such as density and attenuation coefficient output by the equipment; MRI multi-sequence data: sourced from magnetic resonance imaging equipment, core of which includes two types of standard sequence three-dimensional image data: T1-weighted images (T1WI) and T2-weighted images (T2WI), preserving the tissue contrast, signal intensity and other characteristic information of each sequence; Clinical metadata: sourced from the hospital's clinical information system (HIS) / radiological information system (RIS), is structured scalar data, including initial screening information on lesion location such as imaging level and approximate anatomical region, patient physiological parameters such as age, gender, body mass index (BMI), blood pressure, and past medical history tags such as history of tumors, tuberculosis, and cardiovascular diseases, etc. (binarized / classified tags). When standardizing data: Image data resampling: All three-dimensional image data of dual-energy spectral CT data and MRI T1 / T2 weighted images were uniformly resampled using a trilinear interpolation algorithm. The spatial resolution of all image data was calibrated to 512×512×64 (length×width×slice thickness), and the voxel spacing was synchronously normalized to ensure that the spatial dimension of image data under different devices and scanning parameters was completely consistent. At the same time, the gray values ​​of the images were standardized by window width and window level. CT used lung window / mediastinal window, and MRI used the corresponding sequence standard window width, mapping the gray values ​​to the [0,1] value range to eliminate the differences in gray value calibration between devices. For categorical / binarized data, such as gender and past medical history labels, one-hot coding is used. Discrete feature vectorization is performed; all processed clinical metadata is concatenated in a preset order to generate a one-dimensional clinical feature vector with fixed dimensions, ensuring that the clinical metadata feature dimensions of different patients are consistent. When constructing a multimodal tensor matrix: Based on standardized image data and clinical feature vectors, a unified multimodal tensor matrix is ​​constructed using the channel dimension feature stitching method, generating a three-dimensional input tensor with dimensions of 512×512×64×C, where C is the total number of modal channels, and the channel allocation rule is fixed and unique. The channel allocation and splicing method is as follows: Spectroscopic CT dual-energy data: occupying two independent channels, corresponding to a 140kVp high-energy channel and an 80kVp low-energy channel respectively, preserving the differentiated physical characteristics of the two energies; MRI multi-sequence data: occupying two independent channels, corresponding to the T1-weighted image channel and the T2-weighted image channel respectively, preserving the tissue contrast characteristics of multiple sequences; Clinical metadata: Occupying one independent channel, the standardized one-dimensional clinical feature vector is extended to a three-dimensional spatial dimension of 512×512×64 through a broadcast mechanism to achieve full-domain fusion of clinical features and image spatial features; Tensor stitching: Following a fixed channel order of high-energy spectral CT → low-energy spectral CT → MRI T1-weighted image → MRI T2-weighted image → clinical metadata, feature stitching is performed along the channel dimension to generate a standard multimodal input tensor with dimensions of 512×512×64×5, thus completing the unified integration of multi-source data and providing standardized input for the subsequent dynamic enhancement convolution module; The core improvement of this module lies in clarifying the receiving specifications, standardization algorithms, tensor splicing rules, and channel allocation details of multi-source data. It solves the problems of inconsistent formats, insufficient feature integration, and ambiguous channel definitions in traditional multimodal data fusion, ensuring that different types of data are completely standardized in terms of spatial dimension, feature units, and channel allocation. This lays a unified data foundation for feature extraction and fusion analysis in subsequent modules. At the same time, all processing steps are executed automatically end-to-end without manual intervention, making it suitable for actual clinical application scenarios.

[0026] The multimodal data adaptation module performs standardized processing, laying a unified data foundation for the entire process: Precise delivery of output results: The standardized tensor matrix is ​​directly passed to the dynamic enhancement convolution module. The tensor space dimension of 512×512×64, channel allocation rules, CT2 / MRI2 / clinical1, feature dimension range, image [0,1], and clinical normalized [-1,1] are clearly defined. This allows the dynamic enhancement convolution module to perform feature enhancement directly without additional data processing, achieving zero data adaptation cost between steps. Basic information reuse across steps: The initial screening information of lesion location and examination site label in clinical metadata are synchronized to the subsequent lesion morphology constraint module and cross-scale feature fusion module. The lesion morphology constraint module can directly narrow down the extraction range of lesion candidate areas based on the initial screening location, and the cross-scale feature fusion module can match the morphological prior rules of the corresponding lesion based on the examination site, avoiding blindly traversing the entire image in subsequent steps. Globally unified data specifications: Define a unified three-dimensional voxel spacing, grayscale standard, and feature vector dimension for feature calculation in all subsequent steps. This ensures that the convolution calculation of the dynamically enhanced convolution module, the attention weighting of the lesion-background attention module, and the morphological parameter extraction of the lesion morphology constraint module are all based on the same data benchmark, avoiding feature deviations caused by inconsistent data specifications.

[0027] Dynamically Enhanced Convolutional Module: Based on Improved The algorithm introduces an adaptive threshold parameter to adjust the wavelet coefficients. Multi-scale features are extracted by combining three parallel convolutional branches with different dilation rates; Specifically, the adaptive threshold adjustment mechanism enhances images using the following formula:

[0028] in, For enhanced image output values, For the scale hierarchy of wavelet decomposition, As a preset overall decomposition scale, this module is fixed at 4 levels to adapt to the multi-scale feature distribution of medical images. These are wavelet translation parameters used to locate the wavelet basis functions in space. voxel coordinates, For adaptive threshold parameters, for The low-frequency approximation components after level decomposition retain the overall contour, gray-level distribution, and other basic features of the image, avoiding the loss of global information during enhancement. They are dynamically generated from the local contrast of the input tensor. When the local contrast is lower than a preset threshold of 0.15, Automatic enlargement is used to enhance edge features; three parallel dilated convolution branches are set with dilation rates of 1, 3, and 5, respectively, to perform multi-scale preliminary extraction on the enhanced feature map, followed by group normalization. It replaces batch normalization processing, reducing the impact of fluctuations during training with small batches of data; It should be further explained that the formula realizes three-dimensional wavelet multi-scale decomposition of the input multimodal tensor and adaptive coefficient weighted reconstruction. The wavelet decomposition separates the image features into high-frequency detail components corresponding to lesion edges, small textures and low-frequency approximate components corresponding to background tissue and overall contour. Then, the high-frequency components are differentially weighted by the adaptive threshold parameter. Finally, the weighted high-frequency components and low-frequency components are fused to complete the image reconstruction, which not only preserves the basic contour features, but also enhances key details such as lesion edges in a targeted manner. The core of the dynamic enhancement convolution module is to achieve adaptive enhancement and multi-scale preliminary extraction of image features. It solves the problem of blurred edge features in low-contrast areas by introducing an adaptive threshold adjustment mechanism, and achieves synchronous extraction of multi-scale features by combining multi-dilution parallel convolution. At the same time, it optimizes the normalization strategy to adapt to small-batch training scenarios, providing enhanced feature maps with clear features and complete scale for subsequent attention weighting and morphological constraints. For the 3D feature map enhanced by adaptive threshold wavelet filtering, three parallel 3D dilated convolution branches are constructed to achieve synchronous and redundant extraction of multi-scale features. The specific design is as follows: Branch convolution specification: All three branches use 3×3×3 three-dimensional dilated convolution kernels to ensure the spatial correlation of three-dimensional voxel features and adapt to the three-dimensional structural characteristics of medical images. Differentiated settings for the porosity: The porosity of the three branches is fixed at 1, 3, and 5, respectively. Porosity 1 is a standard convolution that extracts local fine features, corresponding to small lesions and edge details; porosity 3 extracts medium-scale contextual features, corresponding to the main lesion area; and porosity 5 extracts large-scale global features, corresponding to the spatial relationship between the lesion and surrounding tissues, achieving full coverage of multi-scale features from local to global. Branch output fusion: The convolutional output feature maps of the three branches maintain the same spatial resolution of 512×512×64 as the input. Multi-scale feature fusion is achieved by concatenating the channel dimensions. The number of feature channels after fusion is three times that of the original single branch, preserving complete multi-scale feature information. Feature distribution calibration is performed by replacing batch normalization with group normalization. The specific implementation details are as follows: Grouping rules: Divide the fused feature channels into groups of 16. If the total number of channels is not divisible by 16, the last group retains the remaining channels to ensure the consistency of feature distribution in each group. Normalization calculation: The mean and variance of all voxel eigenvalues ​​in each group are calculated and mapped to a standard normal distribution with a mean of 0 and a variance of 1, eliminating the differences in the dimensions and distribution of features at different scales. The core advantages of replacing batch normalization: Batch normalization's normalization effect depends on the batch size. In the scenario of training with small sample medical data, the batch size of this system is set to 4. The mean and variance of the batch statistics fluctuate greatly, which can easily lead to unstable model training and decreased feature extraction accuracy. In contrast, group normalization is based on intra-channel group statistics and does not depend on the batch size. Even when training with small batches, it can maintain a stable normalization effect, effectively reducing the impact of fluctuations in training with small batch data and improving the model's generalization ability in small sample scenarios.

[0029] After normalization, the module outputs a multi-scale fused feature map with complete scale, enhanced features, and standard distribution, which serves as the input to the lesion-background attention module, completing the feature processing flow of this module.

[0030] The dynamic enhancement convolution module directly calls the standard tensor matrix output by the multimodal data adaptation module and adopts differentiated enhancement strategies for different channel features in CT / MRI / clinical imaging: focusing on edge enhancement and multi-scale convolution for CT / MRI image channels, while preserving the original feature distribution of clinical metadata channels to avoid loss of clinical information caused by meaningless enhancement; In conjunction with the lesion-background attention module: Output customized feature map: The multi-scale fused feature map, maintaining a spatial dimension of 512×512×64 and a channel count three times that of a single branch, is passed to the lesion-background attention module. The feature map has already completed edge enhancement of low-contrast areas and integration of multi-scale features, so that the attention mechanism of the lesion-background attention module no longer needs to process blurred features, and directly performs spatial + channel weighting on high signal-to-noise ratio features, thereby improving the accuracy of attention localization. Synchronous multi-scale feature identification: The local / medium-scale / global feature region identifications corresponding to the three porosity ratios of 1 / 3 / 5 are synchronized to the lesion-background attention module. The lesion-background attention module can adopt differentiated attention weight calculation strategies for feature regions of different scales, such as increasing the spatial attention weight for local fine feature regions and increasing the channel attention weight for global contextual feature regions. Cross-step coordination with the cross-scale feature fusion module: The number of channels and feature dimension information of multi-scale feature extraction are synchronized to the cross-scale feature fusion module, providing a basic dimension reference for the construction of four feature scale layers in the cross-scale feature fusion module, and avoiding the disconnect between the scale layer construction of the cross-scale feature fusion module and the previous multi-scale feature extraction.

[0031] Lesion-background attention module: It adopts a cascaded structure of spatial attention and channel attention, calculates channel dependence through feature cloud dot product attention, and generates a fusion weight matrix to enhance lesion features; Specifically, the spatial attention submodule performs global average pooling and global max pooling on the feature map output by the dynamically enhanced convolutional module. The results are concatenated and then generated through a 1×1 convolution to produce a spatial attention map. The weights of the lesion region are calculated using the following formula:

[0032] in, Weighted by the lesion area. for Activation function This is a 1×1 convolution operation. Input feature map; : The result of three-dimensional global average pooling performed on the input feature map F, with dimensions 1×1×1×K; : The result of three-dimensional global max pooling performed on the input feature map F, with dimensions 1×1×1×K; Channel-dimensional concatenation operation concatenates two 1×1×1×K pooled features into a 1×1×1×2K fused feature; The core function of the formula is to integrate the global distribution information of global average pooling with the local peak information of global max pooling, so that the model can accurately learn the spatial location features of lesions. The generated spatial attention map can specifically enhance the feature weights of the lesion area and initially suppress the interference of background tissue. Channel attention submodule: Based on a nonlocal shape analysis mechanism, it constructs a voxel-level feature cloud from the feature map. ,in, For the number of voxels, The number of feature channels; Calculate inter-channel dependencies using dot product attention:

[0033] The output is a channel attention weight matrix with dimensions C×N, where each element represents the attention weight of the corresponding channel to the corresponding voxel. for The activation function normalizes the correlation strength between channels, ensuring that the sum of the weights for each channel is 1, thus guaranteeing the probabilistic interpretability of the weights. Feature Cloud The transpose of the matrix, with dimensions C×N, is used to achieve the dimensionality conversion between channels and voxels; The correlation matrix between channels is calculated, and the result has a dimension of C×C, where each element represents the feature correlation of the channel. It is a scaling factor used to alleviate the scaling issues caused by the number of channels in dot product operations. To address the issue of numerical overflow caused by excessively large values ​​and improve computational stability; Activation function, its purpose is to activate the transposed feature cloud. Perform nonlinear transformations to enhance effective features and suppress invalid noise; The spatial attention map and the channel attention map are multiplied element by element to generate a fusion weight matrix, which is used to weight and enhance the original feature map to highlight the features of the core lesion region. The core function of the formula is to learn which channels' features are better able to characterize lesions by calculating the feature correlation between channels. For example, the feature channels of lesion edge are more correlated with the core texture feature channels, and their weights will be significantly increased, while the weights of channels corresponding to background noise will be suppressed, thus achieving accurate selection of feature channels.

[0034] It should be further explained that the lesion-background attention module adopts a cascaded structure of spatial attention and channel attention. The core is to accurately distinguish the lesion area from the background tissue features through a dual-dimensional attention mechanism. First, spatial attention is used to locate the potential location of the lesion, and then channel attention is used to strengthen the dependency relationship of the lesion-related feature channels. Finally, a fusion weight matrix is ​​generated to weight and enhance the original feature map, so as to achieve accurate highlighting of the core features of the lesion area and suppression of background noise. Module input and overall process: The module takes the multi-scale fused feature map output by the dynamically enhanced convolutional module as input, with dimensions of 512×512×64×K, where K is the number of feature channels after fusion. First, the spatial attention submodule generates the weight distribution of the spatial dimension. Then, the spatially weighted feature map is input into the channel attention submodule to generate the weight distribution of the channel dimension. Finally, the two-dimensional attention fusion is achieved by element-wise multiplication, and the weighted enhanced lesion feature map is output. The overall process is: feature input → spatial attention localization → channel attention enhancement → two-dimensional weight fusion → lesion feature enhancement output. The core objective of the spatial attention submodule is to identify the spatial distribution region of lesions in the feature map, generate a spatial attention weight map, enhance the features at the location of the lesions, and suppress the features of the background region, as detailed below: Feature pooling extracts global information: For the input feature map F, with dimensions 512×512×64×K, perform three-dimensional global average pooling and three-dimensional global max pooling operations respectively: Global average pooling: Calculates the average value of all voxel feature values ​​in each channel and outputs an average pooled feature vector with a dimension of 1×1×1×K, preserving the global grayscale distribution information of the feature map; Global max pooling: Calculates the maximum value of all voxel feature values ​​in each channel and outputs a max pooling feature vector with a dimension of 1×1×1×K, highlighting local peak features in the feature map, corresponding to high-response areas such as lesion edges and cores; Pooling Feature Fusion and Attention Map Generation: The average pooling feature vector and the max pooling feature vector are concatenated along the channel dimension to obtain a fused feature vector of dimension 1×1×1×2K. A 1×1×1 three-dimensional convolution operation is performed on the fused feature vector to compress the number of channels back to K, achieving dimensionality reduction and information integration. Finally, an activation function is applied... Mapping the convolution output to the [0,1] range generates dimensions and input features. Figure 1 Spatial attention map 512×512×64×K, where the closer the value is to 1, the higher the probability that the spatial location is a lesion, and the closer it is to 0, the higher the probability that it is background.

[0035] Pre-processing with the dynamically enhanced convolutional module: Receives the multi-scale enhanced feature map output by the dynamically enhanced convolutional module, and adapts the attention calculation logic based on the multi-scale attributes of the feature map: For local fine features with a hole rate of 1, spatial attention calculation is performed to accurately locate small lesions; for global features with a hole rate of 5, channel attention calculation is performed to filter global feature channels related to lesions, achieving accurate adaptation of the attention mechanism and multi-scale features; Post-implementation coordination with the lesion morphology constraint module: Output weighted feature map + attention weight mask: The feature map after spatial + channel fusion weighting is passed to the lesion morphology constraint module, and the attention weight mask is synchronized at the same time. The weight ≥0.5 is the high probability region, and <0.5 is the background region. The lesion morphology constraint module directly extracts the lesion candidate region based on the mask, without the need to perform morphological segmentation on the whole image, thus avoiding the interference of the background region on the extraction of morphological parameters. Synchronize lesion spatial positioning information: synchronize the three-dimensional core coordinates of the lesion identified by spatial attention to the lesion morphology constraint module. The lesion morphology constraint module extracts morphological parameters around these coordinates, further narrowing the calculation range and improving the accuracy of parameter extraction. Cross-step coordination with cross-scale feature fusion module The list of effective feature channels after channel attention filtering is synchronized to the cross-scale feature fusion module. When constructing the scale layer, the cross-scale feature fusion module can prioritize retaining these effective channel features, reduce the computation of invalid channels, and improve the efficiency of cross-scale fusion.

[0036] Lesion morphology constraint module: Extracts shape parameters, roundness, lobulation index and texture parameters of lesion candidate regions, performs dynamic constraint filtering based on a preset threshold library, generates a calibration feature map of semantic features + morphological features, and works in collaboration with lesion-background attention module and cross-scale feature fusion module. Specifically, the lesion morphology constraint module, as the core module for feature selection and optimization, constructs a constraint mechanism based on prior knowledge of clinical lesion morphology, such as the fact that lung cancer lesions are mostly lobulated and brain metastases are mostly round. Morphological feature extraction: Morphological analysis is performed on the attention-weighted feature map to extract the geometric parameters of the lesion candidate region, including: Shape parameter: roundness , For the area, Perimeter; It reflects the degree of similarity between the lesion area and an ideal sphere. The closer the value is to 1, the closer the shape is to a circle; the smaller the value, the more irregular the shape, such as lobed or cord-like. The core function of the formula is to quantify the roundness of lesions, which is a key indicator for distinguishing different lesion types. For example, brain metastases are mostly round. Approximately 1; lung cancer is mostly lobulated. Too small; Fragmentation Index It reflects the degree of lobulation at the edge of the lesion; the higher the value, the more obvious the lobulation; the closer the value is to 1, the smoother the edge. 3D calculation formula:

[0037] in, : The maximum three-dimensional diameter of the candidate lesion region, in units of voxel spacing × voxel number, is obtained by calculating the maximum Euclidean distance between any two points in the region; The equivalent sphere diameter of the lesion candidate region, in units of voxel spacing × voxel number, derived from volume. It is derived that ; The core function of the formula is to quantify the lobulation characteristics of lesion edges, which helps differentiate lung cancer, which is characterized by pronounced lobulation. Larger, benign nodules with smooth edges. The core indicator is close to 1; Texture parameters: contrast and entropy values ​​of the gray-level co-occurrence matrix (GLCM), reflecting the density uniformity within the lesion; Dynamic constraint filtering: A morphological constraint threshold library is constructed, with preset threshold ranges for different lesion types, such as a roundness of 0.3-0.7 and a lobulation index ≥1.2 for lung cancer; and a roundness of 0.7-0.9 and a lobulation index ≤1.1 for brain metastases, to filter candidate regions. When the morphological parameters of the candidate region fall within the threshold range of the corresponding lesion type, its feature weight is retained and enhanced, and the weight coefficient is increased to 1.2-1.5; When the morphological parameters exceed the threshold range and the deviation is ≥30%, it is marked as a suspected interference region, and its feature weight is reduced, with the weight coefficient reduced to 0.5-0.8; Feature calibration output: The filtered feature map is concatenated with the morphological parameter feature vector to generate a calibration feature map with a 20% increase in dimension, which is then passed to the cross-scale feature fusion module to achieve dual constraints of semantic features and morphological features. It should be further explained that, based on the attention-weighted feature map, the candidate lesion region is first extracted by adaptive threshold segmentation, and the background region with attention weight < 0.5 is excluded. Then, the three-dimensional morphological and texture features of each candidate region are quantitatively extracted. All parameters are calculated based on three-dimensional voxel data to adapt to the three-dimensional structural characteristics of medical images. Synergy with other modules: Collaboration with the lesion-background attention module: The attention module provides spatially and channel-weighted high signal-to-noise ratio semantic features, enabling the morphological constraint module to focus only on high-probability candidate regions, avoiding indiscriminate computation of the entire image and improving efficiency; In collaboration with the cross-scale feature fusion module: the calibration feature map supplements the quantified morphological-texture features, enabling cross-scale fusion to not only focus on the multi-scale adaptation of semantic features, but also to combine the rationality of morphological parameters to further improve the accuracy of lesion identification. For example, if the morphological parameters of small-scale semantic features meet the threshold of malignant lesions, they will be given priority enhancement.

[0038] The calibration feature map is finally output to the cross-scale feature fusion module, providing semantically effective and morphologically matched high-quality input for subsequent multi-scale feature integration, and significantly reducing false positive interference.

[0039] Cross-scale feature fusion module: Constructs 4 feature scale layers, and fuses features through dynamic weights jointly adjusted by information entropy and morphological matching degree, preserving lesion details and global context information; Specifically, the cross-scale feature fusion module employs an improved Feature Pyramid Network (FPN), which works in deep collaboration with the calibration feature map output by the lesion morphology constraint module, with four feature scale layers at resolutions of 1 / 2, 1 / 4, 1 / 8, and 1 / 16. Low-level feature enhancement: At half the resolution of the low-dimensional feature layer, residual connections and morphological constraint weights are introduced to preserve the edge details and morphological features of small lesions. High-level feature optimization: At a high-dimensional feature layer with a resolution of 1 / 16, 3D contextual information is extracted through 3D convolution, and semantic-morphological fusion lesion volume features are constructed by combining morphological parameters. Feature fusion mechanism: Features at different scales are fused by using dynamic weights through parallel connection layers. The weight parameters are jointly adjusted by the information entropy and morphological matching degree of each layer's features. The information entropy weight accounts for 0.6, and the morphological matching degree weight accounts for 0.4. The higher the information entropy and the better the morphological matching degree, the larger the feature layer weight coefficient, with a value range of 0.1-0.4. Output processing: Global average pooling is performed on the fused features, and the class probability vector is output through a fully connected layer. The preliminary identification results are obtained by non-maximum suppression (NMS) filtering, which includes lesion location, class, and confidence level. It should be further explained that the semantic-morphological calibration feature map output by the core and lesion morphology constraint module is deeply coordinated. Through four feature scale layers with different resolutions, it achieves full feature coverage from local details to global context. It innovatively introduces a dual-factor dynamic weight mechanism of information entropy + morphological matching degree to solve the problem of lesion detail loss or background interference caused by the fixed weight fusion of traditional FPN, and finally outputs accurate preliminary recognition results. Lowest feature layer 1 / 2 resolution: Core focus: Focus on small lesions, such as the edge details, local texture and morphological features of lung micronodules with a diameter of <5mm, to avoid the dilution of small lesion features by downsampling; Mid-to-low layer feature layer 1 / 4 resolution: Core positioning: Connecting the underlying details with the mid-level semantics, focusing on the structural features of the main lesion area, such as the outline and lobulation of the lesion; Mid-to-high level feature layers at 1 / 8 resolution: Core localization: Extracting mesoscale contextual features of lesions and associating lesions with adjacent anatomical structures, such as the proximity of blood vessels and bronchi; High-level feature layer 1 / 16 resolution: Core positioning: Extract global contextual information and lesion volume features to construct a semantic-morphological fusion global representation to support lesion category judgment.

[0040] Feature information entropy: measures the information richness of a feature map. The larger the value, the more effective information such as details and edges the feature contains. For example, the information entropy of lower-level features is usually higher than that of higher-level features. Morphological matching degree: measures the degree of matching between the current scale layer features and the prior lesion morphology. The larger the value, the more the region corresponding to the feature matches the morphological features of the target lesion. Synergy with other modules: Collaborating with the lesion morphology constraint module: Receives semantic-morphological calibration feature maps, and incorporates morphological priors into the cross-scale fusion process through morphological matching factors to ensure that the fused features contain both semantic information and conform to morphological logic; Collaborating with the multi-model integrated decision module: The initial identification results output, including location, category, and confidence level, provide basic data for integrated decision-making, and the identification reliability is further improved through multi-model voting. The core improvement of this module lies in breaking through the limitations of the fixed weights of the traditional FPN. It achieves feature fusion that emphasizes both details and the overall picture, and coordinates semantics and morphology through a two-factor dynamic weighting mechanism. This ensures that the detailed features of small lesions are not lost, while suppressing false positive interference through morphological matching, providing high-precision preliminary identification results for subsequent integrated decision-making.

[0041] Multi-model ensemble decision module: based on ResNet50, U-Net constructs three base models and uses a confidence-weighted voting + threshold calibration strategy to output recognition results, forming a collaborative closed loop with the self-supervised optimization module; Specifically, the multi-model ensemble decision module is used to improve the reliability of the recognition results, forming a synergistic closed loop with the cross-scale feature fusion module and the self-supervised optimization module: Base model construction: Three independent base models are built based on different backbone networks, including: Using ResNet50 as the backbone network, focusing on local feature extraction; :by For the backbone network, the focus is on global context modeling; Using U-Net as the backbone network, focusing on lesion boundary segmentation; Model training collaboration: The pre-trained weights of the self-supervised optimization module are used to initialize three base models, and complementary data augmentation strategies are adopted in the fine-tuning stage; Using rotation and flipping, Use partial occlusion, Gamma correction is used to avoid model overfitting; Integrated decision-making mechanism: Employing a strategy of confidence-weighted voting + threshold calibration. ① Receive the preliminary identification results from 3 base models and extract the lesion category probability and confidence level output by each model; ② Calculate the weighted voting score: ,in, For the first The class probabilities of each model These are the model weights, dynamically allocated based on the accuracy of the validation set. Higher accuracy results in larger weights, with a total of 1. ③ Set dynamic decision threshold: When the weighted voting score is ≥0.85, the recognition result is directly output; when the score is between 0.6 and 0.85, the morphological parameters of the lesion morphology constraint module are called for secondary verification. If the morphological matching degree is ≥0.7, the result is confirmed; otherwise, it is marked as a suspected lesion; when the score is <0.6, "no clear lesion" is output. Results Feedback: The integrated decision results are fed back to the self-supervised optimization module as a basis for hard sample mining and to continuously optimize the weight allocation of the base model; It should be further explained that the multi-model integration decision module, as the core module for enhancing the reliability of the identification results, is based on the design concept of complementary heterogeneous backbone networks + dynamic weighted integration + secondary validation in clinical trials. It constructs three functionally differentiated base models and integrates the outputs of multiple models through confidence-weighted voting and threshold calibration strategies. At the same time, it forms a collaborative closed loop with the cross-scale feature fusion module and the self-supervised optimization module to solve the problem that single models are easily affected by data distribution bias and feature extraction bias. Finally, it outputs high-precision and high-reliability lesion identification results. Complementarity explanation: Focusing on local details, such as the edges of tiny lesions, Capture global correlations, such as the locational relationship between lesions and blood vessels. Accurately locate the lesion area, reduce background interference, and achieve complementary feature extraction from three dimensions: local, global, and regional. Heterogeneous model complementarity: Three base models extract features from three dimensions: local, global, and regional, covering complex lesion features that are difficult for a single model to capture, thus improving recognition robustness; The decision-making mechanism is precise: weighted voting combines confidence and model accuracy, and dynamic threshold calibration combines morphological priors, providing double assurance for the reliability of the identification results, and the false positive rate is significantly reduced compared to a single model; Collaborative closed-loop optimization: It works in conjunction with the self-supervised optimization module to continuously update model weights and fine-tune the model through clinical feedback, enabling the system to adapt to clinical data from different hospitals and equipment, thereby improving its adaptability to practical applications.

[0042] Self-supervised optimization module: Optimizes the model through unlabeled data pre-training, contrastive learning fine-tuning and hard sample mining, and dynamically updates the model weights and morphological constraint threshold library by combining the decision results of multi-model integration. Specifically, the self-supervised optimization module is used to solve the problem of training with few samples, and it works in conjunction with the multi-model ensemble decision module for optimization. Pre-training phase: Self-supervised training is performed using unlabeled public medical image datasets, such as LUNA16 and BraTS2021. Data augmentation strategies such as image rotation, flipping, and local occlusion are adopted to train the feature extraction backbone network of the three base models. Fine-tuning phase: Combine the pre-trained model with a small number of labeled target lesion data (≥30 cases), and introduce a contrastive learning loss function:

[0043] in: The contrastive learning loss value ranges from [0, +∞). The smaller the loss value, the stronger the model's ability to distinguish between similar and dissimilar features. For cosine similarity, This is a temperature parameter, with a value of 0.07. These are feature vectors of similar lesions; The negative sample feature vectors include feature vectors of other lesion types and feature vectors of background tissue. The size of the negative sample set is set to 128 to ensure the effectiveness of the comparison. The temperature parameter, fixed at 0.07, primarily functions to adjust the discriminative power of similarity. The smaller the value, the more rigorous the model's distinction of similarity, and the more accurately it can differentiate between subtle differences in lesion features and background features. All negative samples and positive samples The similarity index, adjusted by the temperature coefficient, is used as the denominator for normalization. The core function of the formula is to force the model to learn the feature representations of lesions of the same type that are highly similar and lesions of different types / background that are highly different. This allows the model to still discover the core feature patterns of the target lesion even with small sample data, solving the problem of overfitting and weak feature discrimination ability of traditional supervised learning in small sample scenarios, and greatly improving the model's recognition accuracy of the target lesion. False positive suppression: Construct a false positive sample library containing easily confused regions such as vascular calcification and inflammatory hyperplasia, combine samples of suspected lesions labeled by a multi-model integrated decision module, and optimize the classifier through a hard sample mining strategy to reduce the false positive rate; Model weight updates: Regularly adjust the ensemble weights of the three base models based on the correction results from clinical feedback and the correct labels after physician review, to ensure that the system adapts to changes in the distribution of clinical data; It should be further explained that the self-supervised optimization module, as the core support module for improving the generalization ability and continuous iteration of the entire system model, focuses on solving key problems in medical imaging lesion recognition scenarios, such as scarce labeled data, poor generalization of small sample training, high false positive rate, and decay of model adaptability with changes in clinical data. Through a four-layer optimization mechanism of unlabeled data pre-training + comparative learning small sample fine-tuning + hard sample mining optimization + multi-dimensional dynamic update, it forms a deeply collaborative closed-loop optimization system with the multi-model integration decision module. At the same time, it realizes dual-dimensional dynamic iteration of model weights and morphological constraint threshold library, ensuring that the system still has high-precision recognition ability in small sample scenarios and can continuously adapt to changes in the distribution of actual clinical data. Pre-training dataset construction: Integrate multimodal and multi-site publicly available unlabeled medical image datasets, covering the image types and lesion sites adapted to the system, including LUNA16 chest CT, BraTS2021 brain MRI, CBIS-DDSM mammography, LITS abdominal CT, etc., with a single dataset sample size of ≥1000 cases and a total unlabeled sample size of ≥5000 cases, to ensure the comprehensiveness of general features; Self-supervised training task design: Targeting the 3D structural characteristics of medical images, a dual self-supervised task of 3D voxel contrast and masked feature reconstruction is designed to force the model to uncover the intrinsic feature relationships within the images. 3D voxel contrast: Generate two different data augmentation versions of the same image, allowing the model to learn to distinguish the augmentation features of the same image from the augmentation features of other images; Mask feature reconstruction: In a random masked image, 15% to 20% of the voxel regions are used to allow the model to reconstruct the voxel values ​​of the masked regions based on the surrounding features, thereby enhancing the learning of spatial context features. Data augmentation strategy: An unsupervised augmentation strategy matching the distribution of clinical images is adopted, including random rotation ±20°, multi-directional flipping, horizontal / vertical / axial, local voxel occlusion of 8×8×8 voxel blocks, and slight perturbation of gray values ​​±5%, to avoid the enhanced images deviating from clinical reality, while improving the model's ability to adapt to image variations. Training implementation details: The AdamW optimizer was used, with an initial learning rate of 0.001, cosine annealing decay strategy, batch size of 8, and 100 training iterations. The front-end feature extraction backbone networks of the three base models were pre-trained uniformly, and the pre-trained weight files were retained to provide an initialization basis for the subsequent fine-tuning stage. False positive suppression and hard sample mining optimization mechanism: The core objective is to address the key pain point of high false positive rate in medical imaging lesion identification. By constructing a dedicated false positive sample library and combining feedback samples from the multi-model integrated decision module, a hard sample mining strategy is adopted to optimize the model classifier, enabling the model to learn to distinguish between true lesions and easily confused false positive regions, thereby reducing the misjudgment rate from the feature level.

[0044] Construction of a multi-source difficult sample library: A dynamically expandable three-dimensional difficult sample library is constructed, containing two core sample types, both of which are finely annotated as false positives / suspected lesions + confusion types. Basic false positive samples: normal tissue / benign lesion areas that are easily misjudged as lesions in clinical practice, including vascular calcification, increased lung markings, inflammatory hyperplasia, liver cysts, breast fibroadenosis, etc., with ≥50 samples for each type of confusion to ensure coverage of the main interference types; Feedback on suspected samples: Samples fed back in real time by the multi-model integrated decision module, including samples marked as suspected lesions, samples with weighted voting scores between 0.6 and 0.85 and morphological matching degree <0.7, and model prediction results that are determined to be false positives after review by doctors, so as to realize the dynamic accumulation of difficult samples; Hard sample mining strategy: A dual strategy of hard negative sample mining and hard sample priority training is adopted to improve the model's ability to identify hard samples. Hard negative sample mining: For samples in the training batch, calculate the loss value predicted by the model, select the top 30% of samples with the highest loss values ​​as hard negative samples, which are the samples that the model has the most difficulty in distinguishing, and focus on reinforcement learning. Difficult sample priority training: 40% of the samples in the difficult sample library are incorporated into each training batch, and the remaining 60% are regular labeled samples, to ensure that the model focuses on the feature differences of easily confused regions during training; Classifier targeted optimization: The output layer classifiers of the three base models are individually fine-tuned. A center loss is introduced to enhance the compactness of intra-class features, making the features of true lesions and false positive regions farther apart in the feature space, thereby further improving the classifier's discrimination accuracy. The fine-tuning iteration is carried out for 15 rounds with a learning rate of 0.00001 to avoid damaging the trained feature extraction layer.

[0045] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0046] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An automatic lesion recognition system for medical images based on convolutional neural networks, characterized in that: include Multimodal data adaptation module: Receives dual-energy spectral CT, multi-sequence MRI, and clinical metadata, and constructs a 512×512×64×C-dimensional multimodal tensor matrix through linear interpolation, resampling, and normalization; Dynamically Enhanced Convolutional Module: Based on Improved The algorithm uses an adaptive threshold to adjust the wavelet coefficients and combines three parallel convolutional branches with different dilation rates to extract multi-scale features. Lesion-background attention module: It adopts a spatial and channel attention cascade structure, calculates channel dependence through feature cloud dot product attention, and generates a fusion weight matrix to enhance lesion features; Lesion morphology constraint module: Extracts the shape and texture parameters of lesion candidate areas, dynamically filters them through a preset threshold library, generates semantic + morphological calibration feature maps, and works in conjunction with attention and cross-scale fusion modules; Cross-scale feature fusion module: Constructs 4 feature scale layers, and fuses features through dynamic weights jointly adjusted by information entropy and morphological matching degree, preserving lesion details and global context information; Multi-model ensemble decision module: based on ResNet50, Three base models were constructed with U-Net, and the output results of confidence-weighted voting + threshold calibration were used to form a collaborative closed loop with the self-supervised optimization module; The self-supervised optimization module optimizes the model through unlabeled pre-training, contrastive learning fine-tuning, and hard sample mining, and dynamically updates the model weights and morphological constraint threshold library by combining the integrated decision results; the modules are sequentially connected to form an end-to-end recognition framework.

2. The automatic medical image lesion recognition system based on convolutional neural networks according to claim 1, characterized in that: The multimodal data adaptation module receives dual-energy spectral CT imaging data (140kVp high energy, 80kVp low energy); MRI T1 / T2 weighted images and clinical metadata including lesion location screening information, patient physiological parameters, and past medical history tags; linearly interpolates and resamples the image data to a resolution of 512×512×64; normalizes the clinical metadata and converts it into feature vectors; and constructs a 512×512×64×C multimodal tensor matrix by feature splicing.

3. The automatic medical image lesion recognition system based on convolutional neural networks according to claim 2, characterized in that: The dynamically enhanced convolutional module employs an adaptive threshold adjustment mechanism to enhance images using a formula. The adaptive threshold parameter is dynamically generated from the local contrast of the input tensor; when the local contrast is below 0.15, the parameter automatically increases to enhance edge features. Three parallel dilated convolution branches perform preliminary multi-scale extraction of the enhanced feature map, and group normalization is used instead of batch normalization to reduce the impact of training fluctuations on small batches of data.

4. The automatic medical image lesion recognition system based on convolutional neural networks according to claim 3, characterized in that: The spatial attention submodule performs global average pooling and global max pooling on the feature map output by the dynamically enhanced convolutional module. The concatenated result is then convolved with a 1×1 convolution to generate a spatial attention map, and the weights of the lesion region are calculated according to the formula. The channel attention submodule constructs a voxel-level feature cloud based on a nonlocal shape analysis mechanism. ; The dependencies between channels are calculated using dot product attention; the spatial and channel attention maps are multiplied element-wise to generate a fusion weight matrix, which is used to weight and enhance the original feature map, highlighting the features of the core lesion region.

5. The automatic medical image lesion recognition system based on convolutional neural networks according to claim 4, characterized in that: The lesion morphology constraint module constructs a constraint mechanism based on prior knowledge of clinical lesion morphology: extracting morphological parameters of lesion candidate areas from the attention-weighted feature map; filtering candidate areas through a preset morphological constraint threshold library for different lesion types; enhancing feature weights if they meet the threshold range; and marking areas that deviate from the threshold by ≥30% as interference areas and reducing their weights. After splicing and filtering, the feature map and morphological parameter feature vector are combined to generate a calibration feature map with a 20% increase in dimension, which is then passed to the cross-scale feature fusion module to achieve dual constraints of semantics and morphology.

6. The automatic medical image lesion recognition system based on convolutional neural networks according to claim 5, characterized in that: The cross-scale feature fusion module adopts an improved FPN, which works in deep collaboration with the calibration feature map of the lesion morphology constraint module to construct four feature scale layers with resolutions of 1 / 2, 1 / 4, 1 / 8, and 1 / 16: the low-dimensional layer introduces residual connections and morphological constraint weights to preserve the details and morphological features of small lesions. The high-dimensional layer extracts three-dimensional contextual information through 3D convolution and combines it with morphological parameters to construct semantic-morphological fusion lesion volume features; The dynamic weights are jointly adjusted by the parallel connection layer with an information entropy of 0.6 and a morphological matching degree of 0.4, and features of different scales are fused, with weight coefficients ranging from 0.1 to 0.

4. After global average pooling of the fused features, the output class probability vector is passed through a fully connected layer, and then NMS is used to filter and obtain preliminary identification results containing lesion location, class and confidence.

7. The automatic medical image lesion recognition system based on convolutional neural networks according to claim 6, characterized in that: The multi-model integrated decision module, together with the cross-scale feature fusion and self-supervised optimization module, forms a collaborative closed loop to improve recognition reliability: based on ResNet50 local features, Three independent base models were built using global context and U-Net lesion segmentation. The models were initialized with self-supervised pre-trained weights, and complementary data augmentation strategies were used in the fine-tuning stage to avoid overfitting.

8. The automatic medical image lesion recognition system based on convolutional neural networks according to claim 7, characterized in that: The multi-model integrated decision module adopts a decision-making process of confidence-weighted voting + threshold calibration, extracts the probability of each model category and calculates a weighted score based on confidence, and outputs the score directly if it is ≥0.85, and performs secondary verification by combining morphological parameters if it is 0.6-0.

85. If the matching degree is ≥0.7, it is confirmed; otherwise, it is marked as suspected. If the matching degree is <0.6, the rule judgment of no clear lesion is output. The decision result is fed back to the self-supervised optimization module as the basis for optimizing the weight of the base model for difficult sample mining.

9. The automatic medical image lesion recognition system based on convolutional neural networks according to claim 8, characterized in that: The self-supervised optimization module collaborates with the multi-model ensemble decision module to address the small-sample training problem: It utilizes unlabeled publicly available medical image datasets, employing data enhancement techniques such as rotation, flipping, and partial occlusion to self-supervised train the backbone networks of three base models; it fine-tunes the model using data from ≥30 labeled target lesions, introducing a contrastive learning loss function; it constructs a false-positive sample library, combining samples of suspected lesions with hard-sample mining to optimize the classifier; and it periodically adjusts the ensemble weights of the base models based on clinical correction results to adapt to changes in data distribution.

Images