Intelligent focus detection and diagnosis system based on multi-modal medical image fusion

Abstract

The invention discloses a focus intelligent detection and diagnosis system based on multi-modal medical image fusion, and relates to the technical field of medical image processing. A molybdenum target image, an ultrasonic image and a magnetic resonance image of the mammary gland of the patient are acquired through the data acquisition module; a data registration module is used for carrying out partition affine and local refinement registration on the multi-modal image by taking the molybdenum target image as a geometric anchor point and combining parameters such as compression force; extracting registered fusion feature data through a data fusion module; generating a candidate focus set by a focus detection module; finally, the risk assessment module calculates a comprehensive risk score through weighting, correction and consistency verification processes based on a plurality of interpretable feature indexes. And the output module generates a visual diagnosis result. According to the invention, intelligent detection and risk assessment of rechecking of breast lesions are realized, and the diagnosis accuracy and clinical reliability are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to an intelligent lesion detection and diagnosis system based on multimodal medical image fusion. Background Technology

[0002] Multimodal medical image intelligent analysis technology is an important development direction for improving the accuracy of breast disease diagnosis, enabling precise detection and risk assessment of lesions. In recent years, with the advancement of medical image analysis technology and artificial intelligence algorithms, multimodal fusion diagnostic solutions have gradually developed towards automation and intelligence. Existing technologies mostly employ feature fusion or decision fusion methods based on deep learning, extracting features or making independent judgments on each modality of image separately, and then integrating the information at the feature layer or result layer.

[0003] However, such methods often focus on the abstract fusion of features, failing to establish a risk quantification system based on prior medical knowledge and with clear clinical interpretability. There is a lack of unified quantitative correlation and integration standards between lesion features reflected by different imaging modalities (such as calcification morphology on mammography, elasticity and stiffness on ultrasound, and hemodynamic characteristics on MRI), leading to insufficient interpretability of the final risk assessment results and making it difficult for clinicians to trace the basis of their judgments. This lack of quantitative standards, especially when dealing with atypical borderline lesions, may affect the reliability and consistency of risk assessment results, thus limiting the in-depth application of intelligent diagnostic systems in clinical practice.

[0004] Therefore, how to achieve risk quantification among multimodal breast images has become a core technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an intelligent lesion detection and diagnosis system based on multimodal medical image fusion.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] This invention discloses an intelligent lesion detection and diagnosis system based on multimodal medical image fusion, comprising:

[0008] The data acquisition module is used to collect mammogram data of the patient's breast and simultaneously acquire the patient's ultrasound and magnetic resonance imaging data.

[0009] The data registration module is used to perform coordinate transformation on the ultrasound image data and magnetic resonance image data based on the pre-positioned outer contour of the breast, the position of the nipple and the chest wall boundary in the mammogram image data, so as to obtain registered image data that is aligned with the mammogram image data.

[0010] The data fusion module is used to fuse the molybdenum target image data with the registered image data, extract multimodal joint features, and generate fused feature data.

[0011] The lesion detection module is used to detect lesion regions based on the fused feature data and generate a set of candidate lesions;

[0012] The risk assessment module is used to extract multiple preset interpretable feature indicators from the candidate lesion set, and perform quantitative processing and correlation analysis based on the multiple interpretable feature indicators to obtain the comprehensive risk score of the candidate lesion set.

[0013] The output module is used to generate and output the comprehensive risk score into a visual diagnostic result.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0015] 1. This invention effectively overcomes the problems of tissue deformation and spatial misalignment caused by differences in imaging principles and body position by constructing a multimodal fusion framework with mammography images as a unified geometric reference. Through coordinate transformation and registration based on the inherent anatomical structure of the breast, high-precision spatial alignment between ultrasound, MRI, and mammography images is achieved, improving the geometric consistency of multi-source image data in the same coordinate system and enhancing the accuracy of feature fusion and precise lesion localization.

[0016] 2. This invention extracts more stable and consistent fusion feature data to generate a candidate lesion set, enhancing the contrast between small lesions and complex backgrounds. It effectively utilizes the complementary advantages of different imaging modalities: mammography's high resolution for microcalcifications, ultrasound's ability to differentiate cystic and solid masses, and magnetic resonance imaging's ability to capture hemodynamic features. This allows for the simultaneous preservation of morphological and functional information in the fusion features, improving the detection sensitivity for early lesions and atypical lesions.

[0017] 3. This invention enhances the transparency and clinical verifiability of the diagnostic process by introducing a comprehensive risk score calculation and visualization output based on interpretable feature indicators. The system can not only automatically identify suspicious lesions but also provide feature quantification results and risk level judgments with clear physical meaning, assisting doctors in fully understanding the decision-making basis, effectively reducing subjective interpretation differences, and improving the standardization level and reliability of the diagnostic process. Attached Figure Description

[0018] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein:

[0019] Figure 1This is a flowchart of the system steps of the present invention;

[0020] Figure 2 This is a flowchart of the system modules of the present invention;

[0021] Figure 3 This is a detailed flowchart of the data registration module of the present invention;

[0022] Figure 4 This is a flowchart of the lesion detection and multimodal candidate set generation process of the present invention;

[0023] Figure 5 This is a flowchart of the comprehensive risk score calculation and correction process of the present invention. Detailed Implementation

[0024] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0025] Application Overview:

[0026] In existing technologies, multimodal medical image intelligent analysis technology is gradually becoming a key means to improve the accuracy of breast disease diagnosis. With the advancement of deep learning algorithms, current solutions mostly employ feature fusion or decision fusion methods, integrating features extracted independently from each modality of the image. However, these methods suffer from insufficient feature correlation; there is a lack of unified quantitative standards among the calcification morphology, elastic stiffness, and hemodynamic characteristics reflected by different modalities of the image, resulting in poor interpretability of risk assessment results. Especially when dealing with borderline lesions, clinicians find it difficult to trace the diagnostic basis, affecting the reliability of risk assessment.

[0027] To address these issues, research has revealed that the complementarity of different imaging modalities has not been fully explored. For example, mammography is sensitive to calcification but has low soft tissue resolution; ultrasound can assess elastic stiffness but lacks spatial positioning accuracy; and magnetic resonance imaging (MRI) can reflect hemodynamics but suffers from artifact interference. Analysis shows that establishing a cross-modal registration mechanism based on anatomical structures, combined with quantifiable feature indicators, can effectively integrate multi-dimensional information. Further research indicates that constructing an interpretable feature system based on prior medical knowledge can improve the clinical applicability of risk assessment.

[0028] After introducing the basic concept of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0029] Example:

[0030] like Figure 1 As shown, the intelligent lesion detection and diagnosis system based on multimodal medical image fusion includes:

[0031] The data acquisition module is used to collect mammogram data of the patient's breasts and simultaneously acquire the patient's ultrasound and MRI data; it collects mammogram data (left / right MLO and CC views), bilateral ultrasound data, and bilateral MRI data of the patient's bilateral breasts to ensure the feasibility of contralateral site feature mapping.

[0032] The data registration module is used to perform coordinate transformation on ultrasound and magnetic resonance imaging data based on the pre-positioned outer contour of the breast, nipple position, and chest wall boundary in the mammography image data, to obtain registered image data aligned with the mammography image data. The pre-positioning can be automatically identified by image processing algorithms or confirmed by manual annotation by physicians. Coordinate transformation refers to achieving spatial alignment of multimodal images through anatomical landmarks. Specifically, it can be achieved by combining global affine transformation with local linear scaling to ensure spatial consistency of different modal images under a unified coordinate system. Through this step, data from different modalities and imaging principles can be mapped to a unified spatial coordinate system to obtain registered image data aligned with the mammography image.

[0033] The data fusion module is used to fuse mammography image data with registered image data and extract multimodal joint features to generate fused feature data. Fusion feature data extraction refers to integrating complementary information from different images, which can be achieved through feature stitching or weighted fusion algorithms to enhance the integrity of lesion characterization.

[0034] The lesion detection module is used to detect lesion regions based on fused feature data and generate a candidate lesion set; the candidate lesion set includes multiple potential abnormal regions, providing the scope of objects for subsequent risk assessment;

[0035] The risk assessment module is used to extract multiple pre-defined interpretable feature indicators from the candidate lesion set, and to perform quantitative processing and correlation analysis based on these indicators to comprehensively measure the risk level of the candidate lesions and obtain a comprehensive risk score for the candidate lesion set. The comprehensive risk score represents the overall risk level of the lesions. The interpretable feature indicators refer to pre-defined quantitative assessment parameters, such as blood supply characteristics or boundary clarity indicators. Specifically, numerical features can be extracted through image analysis algorithms to provide objective basis for risk assessment.

[0036] The output module is used to generate and output visualized diagnostic results from the comprehensive risk score. The diagnostic results can be presented as image annotations or as numerical reports, providing intuitive evidence for doctors' auxiliary diagnosis.

[0037] Specifically, the system works as follows: Using mammography image data as a baseline coordinate system, a spatial reference frame is established based on the pre-positioned nipple location and chest wall boundaries. Ultrasound and MRI images undergo affine transformation and local correction to eliminate geometric distortions caused by differences in imaging principles. During the fusion process, features of calcification clusters, hypoechoic areas, and dynamically enhanced regions are jointly extracted to form a multidimensional feature vector. The candidate lesion set is generated by merging multimodal detection results, avoiding the risk of missed detections from a single modality. Interpretable feature indicators, through threshold comparison and weight allocation, transform medical experience into a quantitative evaluation model, ultimately generating a visualized diagnostic report.

[0038] Through the above technical solutions, this application achieves spatial consistency fusion of multimodal image data, solving the problem of feature misalignment caused by differences in imaging principles. By constructing an interpretable feature index system, medical experience is transformed into a quantitative assessment model, enhancing the clinical traceability of risk assessment results. The merging strategy of multimodal detection results reduces the probability of missed detections, especially improving the ability to identify atypical lesions. The visualization output module provides doctors with intuitive diagnostic evidence, shortening clinical decision-making time.

[0039] like Figure 2 As shown, this application further proposes that the data acquisition module is also used for:

[0040] Individualized acquisition parameters associated with mammogram data are obtained. These individualized acquisition parameters include at least pressure parameters and exposure parameters. The pressure parameters reflect the degree of physical pressure applied to the breast during imaging, which directly affects breast tissue thickness and image clarity. The exposure parameters are related to image contrast and noise level, and can reflect image quality and detail fidelity.

[0041] The clinically standard range for compression force parameters is 20-80N (data collected in real-time by a pressure sensor on a mammography device, with an accuracy of ±2N), specifically divided into three levels:

[0042] Low pressure: 20-40N (suitable for patients with loose glands or elderly patients);

[0043] Medium compression force: 41-60N (suitable for patients with medium-density glands);

[0044] High compression: 61-80N (suitable for patients with high-density glands or obesity).

[0045] The impact on the correction algorithm and the corresponding handling strategies are as follows:

[0046] Pressure level Changes in breast tissue thickness Sector division correction Scaling factor adjustment Low pressure (20-40N) The tissue is relatively thick (40-60mm). The number of sectors is reduced (4-6 sectors), and the sector angle is increased. Scaling factor 1.2-1.5 (to reduce compression deformation compensation) Medium compression (41-60N) The tissue thickness is moderate (20-40mm). The number of sectors is moderate (6-8 sectors), and the standard angle is used for partitioning. Scaling factor 1.0 (default) High pressure (61-80N) The tissue thickness is relatively thin (10-20mm). Increase the number of sectors (8-10 sectors), decrease the sector angle. Scaling factor 0.7-0.9 (enhanced compression deformation compensation)

[0047] Individual variables relevant to the patient are obtained, including at least the hormone phase parameter. The glandular density and blood supply of breast tissue exhibit dynamic changes under different hormone levels, which can affect imaging findings and lesion identification. Therefore, obtaining these individual variables and utilizing them in subsequent processing enables the system to adapt to individual differences, thereby improving the personalized accuracy of the analysis results. The hormone phase parameter is derived from the number of days (H) from the patient's last menstrual period (LMP) to the examination date, combined with serum estrogen level (E2) measurements.

[0048] Follicular phase: H≤14 days, E2=45-367 pg / mL;

[0049] Luteal phase: 15 days ≤ H ≤ 28 days, E2 = 161-774 pg / mL;

[0050] Perimenopause: H>28 days or E2<45 pg / mL.

[0051] This application introduces individualized acquisition parameters and patient-specific variables, enabling the diagnostic system to adjust and optimize data based on device imaging conditions and the patient's physiological state, thus avoiding detection biases caused by imaging differences or individual variations. Its technical advantages include: not only improving the accuracy of image registration and fusion, but also enhancing the clinical reliability and personalization of risk assessment and diagnostic results, thereby further strengthening the accuracy and practical value of lesion detection and diagnosis.

[0052] like Figure 2 As shown, this application further proposes that the data registration module performs spatial alignment and geometric correction on medical images of different modalities to ensure the accuracy of subsequent fusion and lesion detection, specifically for:

[0053] Based on the pre-defined outer contour of the breast, the position of the nipple, and the boundary of the chest wall, a global affine transformation is performed on the ultrasound and magnetic resonance imaging data to obtain globally aligned data. This step can eliminate the differences in position, orientation, and scale of different modal images on the overall spatial scale, so that the multimodal images are in a unified coordinate system.

[0054] Considering the compression involved in mammography, which causes local deformation of breast tissue, ignoring this factor could lead to registration errors. Therefore, based on the compression force parameter and the local thickness parameter of the breast extracted from the mammography image data, the breast is divided into multiple sectors. For each sector, the global alignment data is subjected to partitioned linear scaling and shearing correction to obtain sector alignment data. This partitioned processing method can accurately reflect the degree of tissue compression in different parts, thereby effectively reducing the error accumulation caused by overall rigidity transformation.

[0055] Sector division is based on the local breast thickness h and compressive force F, and the deformation compensation coefficient is calculated as follows:

[0056]

[0057] in, F represents the compressive force (unit: N), h represents the local thickness (unit: mm), the breast is divided into 36 sectors (each 10° partition), and differential scaling is performed according to the λ value.

[0058] To further improve registration accuracy, the data registration module also calculates the overlap of skin lines and glandular layer contour lines based on the local correspondence between mammogram data and sector alignment data. Based on the calculation results, it performs local fine-tuning of the sector alignment data, ultimately generating registered image data. This process is equivalent to performing fine-grained local corrections on the basis of zonal geometric correction, ensuring consistency across different modalities at the tissue level within the breast.

[0059] This application eliminates deformation errors at different scales layer by layer through phased implementation of global alignment, zonal correction, and local fine-tuning. It is particularly effective in addressing regional tissue stretching caused by pressure by improving registration accuracy through sector division and independent correction. Through the above technical solution, this application effectively solves the problem of anatomical structure misalignment caused by mechanical compression in multimodal images, ensuring the reliability of subsequent multimodal feature fusion and avoiding misjudgment or missed detection of lesions due to registration errors. This provides a precise spatial alignment basis for subsequent lesion detection and risk assessment.

[0060] like Figure 3 As shown, this application further proposes that the lesion detection module specifically includes:

[0061] The first candidate unit is used to detect microcalcification clusters and distorted areas of mass structure in the lesion area through mammography image data, and generate the first candidate lesion set.

[0062] The second candidate unit is used to detect the hypoechoic area and abnormal elasticity area of ​​the lesion area by using the ultrasound image data in the registered image data to generate a second candidate lesion set.

[0063] The third candidate unit is used to detect the dynamic enhancement region of the lesion area by registering magnetic resonance imaging data in the image data and generate a third candidate lesion set.

[0064] The lesion merging unit is used to map the first candidate lesion set, the second candidate lesion set, and the third candidate lesion set to the coordinate system of the mammogram data, and merge them according to positional overlap and morphological similarity to generate a candidate lesion set.

[0065] The lesion detection module employs a multimodal parallel processing and intelligent fusion strategy, specifically implemented as follows: The first candidate unit processes mammography data, utilizing morphological algorithms and texture analysis to detect the distribution characteristics of microcalcification clusters. Simultaneously, structural tensor analysis identifies mass shadows and structurally distorted areas in breast tissue, generating a first candidate lesion set. The second candidate unit processes registered ultrasound image data, identifying abnormal hypoechoic areas based on acoustic impedance characteristics and combining elastography data to detect areas of abnormal tissue hardness. Cross-validation using dual-modal ultrasound features generates a second candidate lesion set. The third candidate unit processes registered magnetic resonance imaging data, analyzing enhancement curve features in dynamic contrast-enhanced time series to capture early enhancement and abnormal washout areas, generating a third candidate lesion set.

[0066] The lesion merging unit maps the three candidate lesion sets to the mammogram coordinate system and uses spatial overlap analysis and morphological similarity measurement algorithms for intelligent merging to eliminate duplicate candidate regions in multimodal detection and finally generate a unified candidate lesion set.

[0067] This approach leverages the advantages of mammography's sensitivity to calcification, ultrasound's sensitivity to tissue elasticity, and MRI's sensitivity to blood flow by independently running three sets of modality-specific detection algorithms. Furthermore, spatial mapping and morphological matching organically integrate multi-dimensional evidence, effectively improving the detection rate of lesions with blurred boundaries. Through this technical solution, this application addresses the problem of missed lesion detection caused by insufficient correlation of multimodal imaging features. The fusion mechanism of independent detection and spatial matching reduces the false positive rate while maintaining the sensitivity of each modality. In clinical practice, it can accurately identify early lesions that only appear in specific images, such as the spatial overlap between areas of abnormal elasticity detected by ultrasound and areas of dynamic enhancement on MRI, which can help determine the invasive characteristics of malignant tumors.

[0068] This application further proposes that several pre-defined interpretable feature indicators include:

[0069] The criteria extracted from the candidate lesion set include at least three of the following: blood supply characteristics, hardness characteristics, calcification and enhancement overlap, boundary clarity, symmetry difference, duct connectivity, and lymphatic correlation.

[0070] Among them, blood supply characteristic indicators refer to parameters obtained by quantifying the signal intensity changes in dynamically enhanced regions in magnetic resonance imaging data. Specifically, they can be achieved by using the peak enhancement rate or area under the curve of the time-signal intensity curve, which is used to reflect the angiogenesis activity in the lesion area.

[0071] Hardness characteristic indicators are parameters quantified using the strain rate ratio or Young's modulus parameter in ultrasound imaging data. Specifically, they can be achieved by using the difference between shear wave velocity and reference tissue, and are used to characterize the degree of abnormality in the mechanical properties of the lesion tissue. Calcification and enhancement overlap indicators are parameters obtained by calculating the spatial overlap between calcification clusters and dynamically enhanced areas in mammograms. Specifically, they can be achieved using the cross-union ratio or overlap area percentage, and are used to assess the spatial correlation between calcified areas and areas with abnormal blood supply. Symmetry difference indicators are parameters obtained by analyzing the differences between lesion features and corresponding features in the contralateral breast. Specifically, they can be achieved using gray-level histogram differences or texture feature distance measures, and are used to identify abnormal asymmetry in unilateral lesions. Duct connectivity indicators are parameters obtained by analyzing the topological relationship between calcification distribution and ductal structure in mammograms. Specifically, they can be achieved using the continuity index of calcification points along the ductal direction or branch density parameters, and are used to assess the likelihood of lesion spread along the ductal system. Lymphatic correlation indicators are parameters obtained by detecting the spatial distance and morphological correlation between the lesion area and the adjacent lymphatic vessels. Specifically, they can be implemented by using the nearest neighbor distance or morphological similarity index of the lymphatic drainage path, and are used to predict the risk of lymphatic metastasis.

[0072] Specifically, in the risk assessment process, at least three interpretable characteristic indicators are first extracted from the candidate lesion set, such as blood supply characteristic indicators, stiffness characteristic indicators, and calcification-enhancement overlap indicators. Blood supply characteristic indicators can be obtained through time-series analysis of dynamic contrast-enhanced magnetic resonance imaging, for example, calculating the slope of signal intensity rise in the lesion area after contrast agent injection; stiffness characteristic indicators can be obtained through ultrasound elastography, for example, measuring the strain ratio between the lesion area and surrounding tissues; calcification-enhancement overlap indicators can be obtained through pixel-level comparison of spatially registered mammograms and magnetic resonance images, for example, statistically analyzing the percentage of overlapping pixels between calcified and enhanced areas. Subsequently, the specific values ​​of each indicator are compared with preset thresholds. For example, when the overlap between calcification and enhancement exceeds 50%, it is judged as a high-risk association, and risk weights are assigned based on the comparison results. Finally, a comprehensive risk score is generated through weighted summation and normalization, enabling the quantitative characteristics of different modalities to be integrated based on a unified standard.

[0073] This approach establishes a quantitative correlation system based on prior medical knowledge, enabling cross-validation of lesion features across different modalities using a unified standard. Independent features from mammography, ultrasound, and MRI images are transformed into clinically interpretable quantitative indicators. For example, the correlation between blood flow abnormalities in calcified areas can be clarified using the overlap index of calcification and blood supply, or unilateral specific lesions can be identified using the symmetry difference index. This multimodal feature integration method ensures that risk assessment results retain the independent diagnostic value of each modality while eliminating the risk of misjudgment from a single modality through quantitative correlation, thereby improving the reliability of comprehensive judgments on borderline lesions.

[0074] like Figure 4 The diagram shown is a flowchart of the comprehensive risk score calculation and correction process for this application. This application further proposes that the risk assessment module perform the following operations:

[0075] Extract the specific values ​​of multiple pre-defined interpretable feature indicators for each candidate lesion from the candidate lesion set;

[0076] The specific value of each explainable feature indicator is compared with the corresponding preset threshold and a comparison result is generated. The value of each explainable feature indicator is compared with the preset clinical threshold, and a corresponding risk weight value is assigned according to the degree of deviation. Among them, malignant related features (such as spiculated edges and rapid enhancement) receive higher weights, while benign related features (such as smooth boundaries and slow enhancement) receive lower weights.

[0077] Based on the comparison results, assign a corresponding risk weight value to each explainable feature indicator;

[0078] The risk weights of all interpretable feature indicators assigned to the same candidate lesion are weighted and summed to obtain the preliminary risk score of the candidate lesion.

[0079] The initial risk score is normalized and corrected based on hormone phase parameters. During the estrogen-dominant period (such as the follicular phase), the score is appropriately reduced to compensate for the false positive risk caused by glandular density. During the progesterone-dominant period (such as the luteal phase), the baseline score is maintained to generate the final risk score of the current candidate lesion.

[0080] The final risk scores of all candidate lesions are summed into a comprehensive risk score for the candidate lesion set.

[0081] Specifically, the risk assessment module assigns a corresponding risk weight value to each interpretable feature indicator based on the comparison results. Let the multiple interpretable feature indicators of the i-th candidate lesion in the candidate lesion set be... The corresponding preset threshold is .

[0082] The formula for assigning risk weights to a single feature is expressed as follows:

[0083]

[0084] in, This represents the specific value of the i-th candidate lesion on the j-th feature index. The function represents the preset threshold for this feature. This is a risk mapping function that can output weight values ​​corresponding to low, medium, and high levels based on the comparison results.

[0085] Then, the risk assessment module performs a weighted sum of all risk weights assigned to the same candidate lesion to obtain a preliminary risk score for that candidate lesion. The calculation formula is:

[0086]

[0087] Where M represents the total number of feature indicators. Preset importance coefficients were assigned to each feature indicator, and the weight coefficients were trained using 5-fold cross-validation with 1000 iterations and a learning rate of 0.01. This was used to reflect the differences in the overall risk assessment among different features. This represents the risk weight of the i-th candidate lesion on the j-th feature.

[0088] Furthermore, considering the influence of individual patient physiological state on breast tissue imaging and lesion manifestation, the risk assessment module normalizes and corrects the preliminary risk score based on individualized hormone phase parameters to generate the final risk score for the current candidate lesion. The formula can be expressed as:

[0089]

[0090] Where H represents the hormone phase parameter, and γ(H) is the hormone phase correction factor, with a value range of [0.9, 1.1], which is dynamically adjusted according to the patient's hormone levels. This is used to standardize risk scores under different hormone cycle conditions, so that imaging data collected from different patients at different time points can still obtain uniform and comparable risk assessment results.

[0091] After completing the risk correction for all candidate lesions, the risk assessment module summarizes the final risk scores of all candidate lesions to obtain the comprehensive risk score of the candidate lesion set, which can be calculated using a weighted average method:

[0092]

[0093] Where N represents the number of candidate lesions, The weighting factor, used to adjust the contribution of different candidate lesions in the overall assessment, can be set according to factors such as lesion volume, location, or duct connectivity.

[0094] This approach transforms different modalities of image features into a unified quantitative system by introducing a specific numerical comparison and weighting mechanism for interpretable feature indicators. Simultaneously, it incorporates hormone phase parameters for individualized correction, ensuring the calculation of risk scores has clear clinical significance and verifiability. This addresses the lack of quantitative standards in multimodal image feature fusion by achieving unified quantification of the risk contribution of different modalities through a dynamic allocation mechanism of preset thresholds and risk weights. Furthermore, the normalization correction of hormone phase parameters eliminates the interference of individual physiological differences in risk assessment, thereby improving the reliability and applicability of the comprehensive risk score in clinical practice.

[0095] This application further proposes that the risk assessment module incorporate an auxiliary diagnostic mechanism based on breast symmetry analysis, specifically for detecting unilateral occult lesions, for the following purposes:

[0096] The symmetric difference index of each candidate lesion in the candidate lesion set is extracted and calculated. The symmetric difference index is obtained by mapping the same site image features of the contralateral breast to the current mammogram coordinate system and then performing difference calculation.

[0097] If the symmetric difference index exceeds the preset first threshold, then the first risk weight value is added to the comprehensive risk score of the current candidate lesion.

[0098] The specific implementation steps are as follows: First, a spatial mapping model of both breasts is established. The region in the mammogram of the contralateral breast that is in a mirror-symmetrical position to the current candidate lesion is accurately mapped to the coordinate system of the current mammogram through affine transformation. This mapping process not only considers the overall outer contour of the breast and the nipple reference point, but also incorporates anatomical features such as the chest wall position for constraint, thereby ensuring the anatomical rationality and spatial accuracy of the mapping.

[0099] After mapping is completed, the system extracts multi-dimensional image features from both the mapped region and the current candidate lesion region. These features include, but are not limited to: gray-level statistical features (such as mean, variance, and skewness), texture features (such as gray-level co-occurrence matrix energy, contrast, and entropy), distribution density features of microcalcification clusters, and tissue structure orientation features (such as Gabor filter orientation response). These features are constructed into feature vectors. (Candidate lesion area) and (Symmetric mapping region). The risk assessment module quantifies the symmetry difference by calculating the Euclidean distance between the two, defined as:

[0100]

[0101] in, This is the symmetric difference index of the i-th candidate lesion. and These are the standardized feature vectors of the current and opposite sides at the same location, including grayscale statistics and texture features.

[0102] In order to perform risk assessment, the system pre-sets a first preset threshold based on large-scale population statistics. First preset threshold Based on statistical analysis of imaging data from 300 bilateral breasts (including 180 cases of benign lesions and 120 cases of malignant lesions), the optimal cutoff value was determined by ROC curve.

[0103] Specific range: 0.35-0.55 (grayscale histogram difference value), where:

[0104] Mild asymmetry (0.35-0.45): Add a low-risk weight (0.8-1.2);

[0105] Significant asymmetry (0.45-0.55): Add a high-risk weight (1.5-2.0).

[0106] When the symmetric difference index Exceeding the first preset threshold This indicates a significant bilateral asymmetry between the current lesion area and the contralateral lesion at the same location, potentially suggesting a risk of occult or early-stage lesions. In this case, the risk assessment module will add a corresponding first risk weight value to the lesion's overall risk score. The specific added weight value... The asymmetry can be dynamically adjusted in stages, as shown in the following formula:

[0107]

[0108] in, These represent the additional risk weights corresponding to mild, significant, and severe asymmetry, respectively. Ultimately, these additional weights are integrated into the lesion risk scoring system through linear superposition, i.e.:

[0109]

[0110] in, This represents the risk score of candidate lesions after the aforementioned threshold comparison, risk weight allocation, and hormone phase correction. This represents the final risk score after considering symmetry analysis.

[0111] This approach, by establishing a bilateral feature mapping and differential computation mechanism, transforms anatomical symmetry differences into quantifiable indicators, enhancing the ability to identify unilateral malignant lesions and overcoming the limitation of lacking bilateral contrast in traditional unimodal analysis. Through this technical solution, this application can effectively detect the risk of malignant lesions caused by unilateral tissue abnormalities, improve the sensitivity of identifying atypical calcifications or small masses in early breast cancer, and enhance the reliability of risk assessment results by quantifying bilateral differences, providing clinicians with clear criteria for judging symmetry abnormalities.

[0112] This application further proposes that the risk assessment module is also used for:

[0113] The calcification and enhancement overlap index of each candidate lesion in the candidate lesion set is calculated. The calcification and enhancement overlap index is obtained by calculating the area intersection-union ratio of the calcification cluster region and the enhancement region of the candidate lesion in the coordinate system of the mammogram data.

[0114] If the overlap index between calcification and enhancement exceeds the preset second threshold, a second risk weight value is added to the comprehensive risk score of the current candidate lesion.

[0115] The risk assessment module integrates calcification-enhancement overlap analysis to detect the characteristic synergy of malignant lesions. This function first operates within the registered multimodal space: based on the coordinate system of the mammogram data, it accurately extracts the calcification cluster regions of candidate lesions (determined through thresholding and morphological filtering), while simultaneously extracting the corresponding spatially located enhancement regions from the registered DCE-MRI images (determined through dynamic enhancement thresholding and a 3D region growing algorithm). The spatial overlap between the two is calculated, using the intersection-to-union ratio (IoU) as a quantitative indicator, which is the area of ​​intersection between the calcified and enhanced regions divided by the area of ​​their union.

[0116] When the overlap between calcification and enhancement exceeds a second preset threshold determined based on retrospective clinical studies, it indicates spatial consistency between calcification distribution and abnormal blood supply. The system then adds a second risk weight to the overall risk score of the current lesion. This weight uses a non-linear incremental strategy: a base weight is added for mild overlap (IoU 0.4-0.5), a moderate weight for significant overlap (IoU 0.5-0.7), and a high weight for high overlap (IoU > 0.7). This weighting is integrated into the risk score calculation process using a feature importance weighting algorithm.

[0117] The specific implementation steps are as follows: In the registered multimodal image space, the system first accurately extracts the calcification cluster regions of candidate lesions based on the coordinate system of the mammogram data. The region was obtained through grayscale thresholding, morphological filtering, and connected component analysis; simultaneously, the corresponding enhanced region was extracted from the registered dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). It is generated through dynamic enhancement threshold determination and three-dimensional region growth algorithm.

[0118] Subsequently, the system calculates the spatial overlap between the calcification cluster region and the enhancement region, using the area intersection-over-union ratio (IoU) as a quantitative indicator, defined as:

[0119]

[0120] in, This represents the intersection area of ​​the calcification cluster and the enhancement region of the i-th candidate lesion. This represents the area of ​​the union of the two sets. This refers to the overlap index between calcification and reinforcement.

[0121] To further correlate risk scores, the system pre-sets a second preset threshold. The threshold value is typically determined based on retrospective clinical studies and falls within the range of 0.4-0.6. (Second pre-set threshold) A retrospective analysis was conducted on the DCE-MRI and mammography fusion data of 150 cases containing calcified lesions, and the intersection-over-union ratio (IoU) of calcified clusters and enhanced areas was calculated.

[0122] Specific range: 0.4-0.6 (area intersection ratio), where:

[0123] Slight overlap (0.4-0.5): Add a base weight value (1.0-1.3);

[0124] High overlap (0.5-0.6): Add a multiplier weight value (1.8-2.2).

[0125] when This indicates a significant spatial consistency between the calcification distribution and abnormal blood supply of the current candidate lesion, suggesting a high-risk characteristic of malignancy. At this point, the system adds a second risk weight value to the overall risk score of the lesion. The additional weights employ a non-linear increasing strategy, as detailed below:

[0126]

[0127] in, These represent the additional risk weight values ​​corresponding to mild, significant, and high overlap, respectively. The weight values ​​are integrated using a feature importance weighting algorithm, which linearly superimposes or weights the weight of calcification-enhancement overlap with the baseline risk score of the aforementioned candidate lesions and other auxiliary indicators (such as blood supply, stiffness, and symmetry difference) to form the final comprehensive risk score for the candidate lesions. :

[0128]

[0129] in, The preliminary risk score is based on threshold comparison and hormone phase correction. Add weights to the analysis of breast symmetry. Add weights to calcification-enhanced overlap, " indicates additional feature weights that can be expanded.

[0130] By introducing calcification and enhancement overlap analysis, the system can comprehensively assess the spatial synergy between abnormal blood supply and microcalcification distribution in lesions on a quantitative basis, making the risk assessment of malignant lesions more accurate and sensitive, especially improving the ability to identify early or occult malignant lesions. Its technical benefits include: enhancing the accuracy of multimodal fusion lesion detection, and making the comprehensive risk scoring system more scientific, personalized, and clinically applicable.

[0131] This application further proposes that the risk assessment module not only performs routine threshold comparisons and risk weighting on the multidimensional imaging features of candidate lesions, but also introduces spatial structural feature indicators closely related to the biological behavior of the lesions to enhance the depth and clinical reference value of the risk assessment. The risk assessment module is also used for:

[0132] Based on ductal connectivity indices and lymphatic correlation indices, the probability of local progression and regional lymphatic metastasis for each candidate lesion in the candidate lesion set is calculated. Among them, the ductal connectivity indices are used to reflect the morphological and functional association between the lesion and the mammary ductal system. By analyzing the degree of connectivity between the lesion boundary and the ductal orientation, the possible spread trend of the lesion within the duct can be inferred. The lymphatic correlation indices are used to assess the spatial relationship between the lesion and adjacent lymph nodes or lymphatic pathways. Combined with enhancement patterns or imaging signal characteristics, the possibility of lymphatic metastasis of the lesion can be inferred.

[0133] The probability of local progression and regional lymph node metastasis are used as weighting factors in the calculation of the comprehensive risk score. This ensures that the final risk assessment result is based not only on static imaging characteristics but also incorporates inferred information about the dynamic evolution trend of lesions. This weighting method allows the comprehensive risk score to more comprehensively reflect the actual risk level of lesions, while highlighting sensitivity to early invasive lesions, thus helping to improve the system's predictive and early warning capabilities in clinical applications.

[0134] Specifically, the risk assessment module is further used for: based on duct connectivity indicators Lymphatic correlation indicators Calculate the local progression probability of each candidate lesion in the candidate lesion set. and regional lymph node metastasis probability .

[0135] Among them, catheter connectivity index This system is used to reflect the morphological and functional relationship between lesions and the mammary ductal system. By analyzing the spatial overlap between the lesion boundary and the ductal orientation, the length of the connecting path, and the density of ductal branches, the system calculates the probability of lesion spread within the ducts.

[0136]

[0137] in, Indicates morphological characteristic parameters of the lesion. The function represents the characteristics of the tissue surrounding the lesion. This can be achieved through weighted regression models or graph network analysis.

[0138] Lymphatic related indicators Used to assess the spatial relationship between the lesion and adjacent lymph nodes or lymphatic pathways, combined with imaging enhancement patterns, blood supply abnormalities, and tissue signal intensity, to infer the likelihood of lymphatic metastasis of the lesion:

[0139]

[0140] in, Enhancement features were observed in the lesions and lymphatic areas. The spatial distance from the lesion to the lymph node, a function This can be achieved through probabilistic graphical models or statistical learning methods.

[0141] In the calculation of the comprehensive risk score, the probability of local progression and the probability of regional lymph node metastasis are used as weighting factors and integrated into the final risk score of candidate lesions. middle:

[0142]

[0143] in, The preliminary risk score is based on threshold comparison and hormone phase correction. and The weights are added based on breast symmetry and calcification-enhancement overlap, respectively. α and β are adjustable weighting coefficients for the probability of local progression and the probability of lymph node metastasis, respectively, to achieve individualized risk assessment.

[0144] By incorporating ductal connectivity and lymphatic correlation analysis, the system can integrate static imaging features of lesions with information on dynamic evolution trends. This allows the comprehensive risk score to not only reflect the morphology and blood supply abnormalities of the lesions but also fully demonstrate the potential risk of spread and metastasis. This multidimensional weighting strategy enhances the system's sensitivity and accuracy for early invasive lesions, resulting in a more comprehensive and interpretable risk assessment with stronger clinical guidance and predictive capabilities. It provides clinicians with a dual-dimensional risk assessment based on clear anatomical evidence, assisting in the development of more precise individualized treatment plans.

[0145] This application further proposes that, after assigning corresponding risk weight values ​​to each interpretable feature indicator based on the comparison results, a consistency check is performed on the relationship between these weight values ​​to ensure that the final risk score calculation not only considers the results of a single feature indicator but also takes into account the overall logical rationality between multiple features. The risk assessment module is also used for:

[0146] Check the consistency among all risk weight values ​​assigned to the same candidate lesion; specifically, perform a combined analysis of the risk weight values ​​of all interpretable characteristic indicators of the same candidate lesion to determine whether they conform to the rules in the preset risk pattern library.

[0147] If a pre-defined combination of high-risk features is detected, such as a lesion having blurred boundaries, rapid dynamic enhancement, and abnormal lymphatic correlation, the lesion will be determined to have a higher clinical risk. A pre-defined multiplicative factor will be applied to the initial risk score of the current candidate lesion to increase its risk level.

[0148] If a preset conflict pattern is detected between risk weight values, such as a lesion appearing as a hypoechoic induration in ultrasound imaging but as stable enhancement in dynamic contrast-enhanced MRI, or contradictory results between duct connectivity indicators and lymphatic correlation indicators, the system will classify the situation as a low consistency pattern and apply a preset confidence reduction factor to the preliminary risk score of the current candidate lesion to reduce the risk of misjudgment caused by inconsistent or uncertain features.

[0149] Specifically, for each candidate lesion i in the candidate lesion set, the system first assigns risk weight values ​​to all its interpretable feature indicators. Perform a combination analysis and determine its consistency based on the pre-defined risk model library R:

[0150]

[0151] in, For consistency scores, the range is typically set to [0,1], and the function... It can be achieved through rule matching, fuzzy logic, or weighted statistical methods to quantify the overall logical rationality among various characteristic indicators of candidate lesions.

[0152] When the system detects a high consistency pattern, meaning that a candidate lesion simultaneously matches a combination of high-risk features from the risk pattern library (e.g., blurred boundaries, rapid increase in dynamic enhancement, and accompanying lymph node abnormalities), the lesion is determined to have a higher clinical risk. At this point, preliminary risk assessment is performed. Applying preset multiplication factors Raise the risk level, risk classification The calculation formula is as follows:

[0153]

[0154] Among them, the preset multiplication factor .

[0155] If a low-consistency pattern is detected, i.e., a combination of conflicting features (e.g., ultrasound imaging shows a hypoechoic induration, but MRI shows stable enhancement, or there is a contradiction between ductal connectivity indicators and lymphatic correlation indicators), the system determines that the lesion has high uncertainty. In this case, preliminary risk assessment... Apply confidence reduction factor Adjust risk score The formula is as follows:

[0156]

[0157] Among them, confidence reduction factor ∈ (0,1).

[0158] By introducing consistency checks and pattern adjustments, the risk assessment process in this application not only achieves quantitative calculation and weighted fusion of multimodal features, but also further enhances the stability and reliability of the results through logical rules and pattern recognition. This application can effectively identify and correct logical contradictions between multimodal features, strengthen the early warning role of high-risk feature combinations, thereby improving the accuracy of risk assessment for borderline lesions and cases with conflicting multimodal information, and providing clinicians with more interpretable and reliable decision-making support.

[0159] The following is a specific embodiment of an intelligent lesion detection and diagnosis system based on multimodal medical image fusion:

[0160] A 45-year-old female patient presented to the breast surgery department of a tertiary hospital after a mammogram revealed "suspicious microcalcifications" in the upper outer quadrant of her left breast. No obvious mass was palpable upon clinical examination. To clarify the nature of the lesion, the radiologist activated this system for multimodal image fusion diagnosis. Through the collaborative analysis of mammography, ultrasound, and MRI, intelligent decision-making support was achieved throughout the entire process, from lesion localization to risk assessment. The data acquisition module simultaneously collected the patient's multimodal images and related parameters: mammography images were taken in medial oblique (MLO) and cephalothorax (CC) views. The pressure sensor recorded a pressure value of 45 N (medium pressure, corresponding to a glandular thickness of 32 mm). Exposure parameters were set to 28 kVp, 65 mAs, and a field of view size of 24 cm × 30 cm. Ultrasound examination showed a 12 mm × 8 mm hypoechoic area in the upper outer quadrant of the left breast, and elastography indicated a shear wave velocity of 3.8 m / s in the lesion area (compared to 1.5 m / s in surrounding normal tissue). The contrast-enhanced imaging sequence showed a rapid rise (peak time 45s) followed by slow washout. The T1-weighted baseline MRI showed a heterogeneous and dense glandular background. The DCE enhanced sequence captured an early enhancement area in the upper outer quadrant of the left breast (enhancement rate 120%). The ductography sequence showed that the lesion was connected to the main duct in the areolar area by branches. Considering the patient's individual variables, her last menstrual period was 20 days before the examination, and her serum estrogen level E2 = 320 pg / mL, which was determined to be the luteal phase (hormonal phase parameter H).

[0161] The data registration module uses mammograms as the global coordinate reference and performs a three-level registration process: First, through global affine transformation, it automatically identifies the nipple point (coordinates (12,8) cm), chest wall boundary (straight line equation y=0.3x+2), and breast outer contour (closed curve), and coarsely aligns the ultrasound and MRI images to the mammogram coordinate system using an affine matrix to eliminate overall offset caused by positional differences; then, it performs sector division correction based on the mammogram compression force F=45N and local glandular thickness h=32 mm, according to the formula... Calculate the deformation compensation coefficient The breast was divided into 36 sectors (each 10° segment), and the hypoechoic area on ultrasound and the enhanced area on MRI were differentially scaled (scaling factor 1.1 for the outer sector and 0.95 for the inner sector). Finally, through local fine-tuning, the overlap between the skin line (grayscale threshold 180) and the glandular layer contour line (CT value -50 to +50HU) in the candidate area of ​​the upper outer quadrant of the left breast (coordinate range 10-14cm × 6-10cm) was optimized (overlap ≥ 85%), so that the alignment error of the lesion boundary of each modality was controlled within 0.8mm.

[0162] The data fusion module integrates the registered multimodal features into a 128-dimensional fusion feature vector, which includes morphological features (area of ​​0.6 cm², perimeter of 3.2 cm, and roundness of 0.72 for mammogram calcification clusters), functional features (peak enhancement rate of 120% for MRI enhancement curves and ultrasound elasticity-hardness ratio of 2.5) and spatial features (center distance between calcification clusters and enhancement areas of 0.3 cm and boundary overlap arc length of 1.8 cm).

[0163] The lesion detection module generates candidate lesions through multimodal parallel detection: mammography uses adaptive threshold segmentation (Otsu algorithm) to identify microcalcification clusters (12 calcification points, cluster density 2.3 / mm²) and structural distortion areas (radial entanglement angle 15°) to generate the first candidate lesion set; in ultrasound detection, the low echo area (echo intensity ratio 0.6) and the area with abnormal elastic hardness (hardness ratio 2.5) have a spatial overlap of 92% to generate the second candidate lesion set; in MRI detection, the early enhancement area of ​​DCE (enhancement rate >100%) is extracted by a three-dimensional region growth algorithm to generate the third candidate lesion set; after multimodal merging, the three-modal candidate lesions are mapped to the mammography coordinate system and merged into a single candidate lesion R1 (coordinates (12.5, 7.8) cm, size 1.3 cm × 0.9 cm) according to IoU ≥ 0.65.

[0164] The risk assessment module extracts five core characteristic indicators from lesion R1 to calculate a comprehensive risk score: blood supply characteristic indicators (MRI enhancement curve slope 2.8, preset threshold 2.0) are assigned risk weights. Risk weights are assigned to hardness characteristic indicators (ultrasonic elastic strain ratio 2.5, preset threshold 2.0). ; Calcification - Enhanced overlap (area intersection-union ratio 0.55, preset threshold 0.5) with added weight Symmetric difference index (difference value of gray-level histogram of the same site in the right breast 0.48, preset threshold 0.45) with additional weight. Risk weights are assigned to the boundary clarity index (boundary gradient standard deviation 120, preset threshold 150). Preliminary risk assessment based on feature importance coefficients. Weighted summation of initial risk Combined with the luteal phase normalization coefficient g(H) = 0.92 (corrected based on patient E2 levels), after hormone phase correction... Finally, combining the duct connectivity index (connection probability 0.75) and the lymphatic correlation index (axillary lymph node short diameter 0.8cm, cortical thickness 0.2cm), the overall risk score was 3.52 (corresponding to BI-RADS 4B).

[0165] The output module generates a layered visualization report: the spatial localization map marks the R1 position in the mammogram CC image, overlaying the ultrasound hypoechoic area (blue), the MRI enhancement area (red), and calcification clusters (yellow), with the spatially overlapping area of ​​the three highlighted in orange; the feature radar map shows the risk contribution percentage of five indicators, including blood supply, stiffness, and overlap, among which calcification-enhancement overlap (32%) and symmetry difference (28%) are the main sources of risk; the decision recommendation is based on the comprehensive risk score and feature consistency, suggesting "ultrasound-guided biopsy", and marking the high-risk feature tracing path. The radiologist observed through the system interface that the lesion in the upper outer quadrant of the left breast showed "polymorphic calcification" on the mammogram, "hypoechoic with hard elasticity" on ultrasound, and "early rapid enhancement" on MRI. The three highly overlapped in a unified coordinate system, and no symmetrical structure was seen at the same site on the contralateral side. Combined with the feature indicators that significantly exceeded the threshold in the risk radar map, the lesion was quickly identified as "moderately suspicious for malignancy". Compared to traditional image reading methods that require switching between different devices for comparison, the system intuitively presents the multidimensional features of lesions through multimodal fusion, avoiding the risk of missed diagnosis due to spatial misalignment. When communicating with patients, the system enhances the consensus between doctors and patients by visually demonstrating the "correlation between calcification and abnormal blood flow". Subsequent pathological biopsy confirmed it as ductal carcinoma in situ, confirming the system's ability to identify early occult lesions.

[0166] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A lesion intelligent detection and diagnosis system based on multimodal medical image fusion, characterized in that: include: The data acquisition module is used to collect mammogram data of the patient's breast and simultaneously acquire the patient's ultrasound and magnetic resonance imaging data. The data registration module is used to perform coordinate transformation on the ultrasound image data and magnetic resonance image data based on the pre-positioned outer contour of the breast, the position of the nipple and the chest wall boundary in the mammogram image data, so as to obtain registered image data that is aligned with the mammogram image data. The data fusion module is used to fuse the molybdenum target image data with the registered image data, extract multimodal joint features, and generate fused feature data. The lesion detection module is used to detect lesion regions based on the fused feature data and generate a set of candidate lesions; The risk assessment module is used to extract multiple preset interpretable feature indicators from the candidate lesion set, and perform quantitative processing and correlation analysis based on the multiple interpretable feature indicators to obtain the comprehensive risk score of the candidate lesion set. The output module is used to generate and output the comprehensive risk score into a visual diagnostic result.

2. The intelligent lesion detection and diagnosis system based on multimodal medical image fusion according to claim 1, characterized in that: The data acquisition module is also used for: Acquire individualized acquisition parameters associated with the molybdenum target image data, the individualized acquisition parameters including at least pressure parameters and exposure parameters; Obtain individual variables related to the patient, including at least hormone phase parameters.

3. The intelligent lesion detection and diagnosis system based on multimodal medical image fusion according to claim 2, characterized in that: The data registration module is specifically used for: Based on the pre-positioned outer contour of the breast, the position of the nipple and the boundary of the chest wall, a global affine transformation is performed on the ultrasound image data and the magnetic resonance image data to obtain globally aligned data. Based on the pressure parameters and the local thickness parameters of the breast extracted from the mammogram data, the breast is divided into multiple sectors, and the global alignment data is subjected to partitioned linear scaling and shearing correction to obtain sector alignment data. Based on the degree of overlap between the skin lines and glandular layer contour lines between the mammogram data and the sector alignment data, the sector alignment data is locally fine-tuned to generate the registered image data.

4. The intelligent lesion detection and diagnosis system based on multimodal medical image fusion according to claim 1, characterized in that: The lesion detection module specifically includes: The first candidate unit is used to detect microcalcification clusters and distorted areas of mass structure in the lesion region through the mammography image data, and generate a first candidate lesion set. The second candidate unit is used to detect the hypoechoic area and abnormal elasticity area of ​​the lesion region through the registered image data, and generate a second candidate lesion set. The third candidate unit is used to detect the dynamic enhancement region of the lesion region through the registered image data and generate a third candidate lesion set. The lesion merging unit is used to map the first candidate lesion set, the second candidate lesion set, and the third candidate lesion set to the coordinate system of the mammogram data, and merge them according to positional overlap and morphological similarity to generate the candidate lesion set.

5. The intelligent lesion detection and diagnosis system based on multimodal medical image fusion according to claim 1, characterized in that: The plurality of preset interpretable feature indicators include: At least three of the following indicators extracted from the candidate lesion set: blood supply characteristic indicator, hardness characteristic indicator, calcification and enhancement overlap indicator, boundary clarity indicator, symmetry difference indicator, duct connectivity indicator, and lymphatic correlation indicator.

6. The intelligent lesion detection and diagnosis system based on multimodal medical image fusion according to claim 5, characterized in that: The risk assessment module performs the following operations: Extract the specific values ​​of multiple preset interpretable feature indicators for each candidate lesion from the candidate lesion set; The specific value of each of the explained feature indicators is compared with the corresponding preset threshold, and a comparison result is generated. Based on the comparison results, a corresponding risk weight value is assigned to each of the explained feature indicators; The risk weights of all interpretable feature indicators assigned to the same candidate lesion are weighted and summed to obtain the preliminary risk score of the candidate lesion. The preliminary risk score is normalized and corrected based on the hormone phase parameters to generate the final risk score of the current candidate lesion. The final risk scores of all candidate lesions are summed into a comprehensive risk score for the candidate lesion set.

7. The intelligent lesion detection and diagnosis system based on multimodal medical image fusion according to claim 5, characterized in that: The risk assessment module is also used for: The symmetric difference index of each candidate lesion in the candidate lesion set is extracted and calculated. The symmetric difference index is obtained by mapping the same site image features of the contralateral breast to the current mammogram coordinate system and then performing difference calculation. If the symmetric difference index exceeds the preset first threshold, then a first risk weight value is added to the comprehensive risk score of the current candidate lesion.

8. The intelligent lesion detection and diagnosis system based on multimodal medical image fusion according to claim 5, characterized in that: The risk assessment module is also used for: The calcification and enhancement overlap index of each candidate lesion in the candidate lesion set is calculated. The calcification and enhancement overlap index is obtained by calculating the area intersection-union ratio of the calcification cluster region and the enhancement region of the candidate lesion in the coordinate system of the mammogram data. If the overlap index between calcification and enhancement exceeds a preset second threshold, a second risk weight value is added to the comprehensive risk score of the current candidate lesion.

9. The intelligent lesion detection and diagnosis system based on multimodal medical image fusion according to claim 5, characterized in that: The risk assessment module is also used for: Based on the duct connectivity index and the lymph node correlation index, the local progression probability and regional lymph node metastasis probability of each candidate lesion in the candidate lesion set are calculated. The probability of local progression and the probability of regional lymph node metastasis are used as weighting factors in the calculation of the comprehensive risk score.

10. The intelligent lesion detection and diagnosis system based on multimodal medical image fusion according to claim 6, characterized in that: After assigning a corresponding risk weight value to each of the explained feature indicators based on the comparison results, the risk assessment module is further configured to: Examine the consistency among all risk weight values ​​assigned to the same candidate lesion; If a preset high-risk feature pattern combination is detected, a preset multiplication factor is applied to the preliminary risk score of the current candidate lesion to increase the risk level of the current candidate lesion. If a preset conflict pattern is detected between the risk weight values, a preset confidence reduction factor is applied to the preliminary risk score of the current candidate lesion.

Images