HER2 positive breast cancer auxiliary prediction method based on multi-modal data fusion

Abstract

The invention relates to the technical field of medical image analysis, and discloses an HER2 positive breast cancer auxiliary prediction method based on multi-modal data fusion. According to the method, preoperative MRI, clinical pathology data and RNA-seq data meeting quality control standards are obtained through dynamic case screening, traditional image features, multi-scale Hessian matrix blood vessel-structure features and topological persistent coherent curvature features are extracted through multi-plane segmentation, and key features are obtained through three-level screening. A multi-factor logistic model fusing radiomics, clinical features and immune scoring is further constructed, a pCR probability value and a feature contribution heat map are output, and MRI data quality control, real-time analysis and interactive report generation are achieved through a matched computer system. The method breaks through the limitation of single modal analysis, dynamically associates the image features with the molecular mechanism, and provides interpretable decision support for precision medical treatment.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and particularly to an auxiliary prediction method for HER2-positive breast cancer based on multimodal data fusion. Background Art

[0002] At present, the gold standard for clinically evaluating the efficacy of neoadjuvant therapy (NAT) for breast cancer is based on the Miller & Payne histological grading system. Pathologists perform pathological evaluations on the surgical specimens of patients who have completed NAT. However, pathological evaluations have invasiveness and latency, and cannot accurately predict the efficacy of NAT treatment early. As a non-invasive examination method, imaging plays an increasingly important role in the evaluation of neoadjuvant therapy. Imaging evaluations are required for all patients at baseline, during neoadjuvant therapy, and after neoadjuvant therapy. Among them, the evaluation of the efficacy of the primary focus needs to cover ultrasound and mammography. For patients who need breast conservation, breast magnetic resonance imaging examination is strongly recommended. Imaging evaluation methods at baseline before NAT treatment include mammography, ultrasound, MRI, etc. Among them, breast MRI is the preferred evaluation method recommended by the guidelines. For the evaluation of the primary focus of breast cancer, both mammography and US are indispensable. For patients who need downstaging for breast conservation, breast MR examination should be routinely recommended. At the same time, this consensus also recommends the follow-up time for imaging evaluation during NAT, and it is recommended to evaluate the primary focus and regional lymph nodes by US and / or breast MRI every 2 courses. To sum up, current breast imaging examinations are the main methods for clinically evaluating the efficacy of NAT before surgery, during surgery, and after treatment, including conventional mammography techniques, ultrasound, MRI, etc. Among them, breast MR examination is the first choice recommended by the NCCN guidelines for breast NAT. In recent years, the widespread application of technologies including mammography techniques, ultrasound, MRI, etc. has provided a non-invasive and relatively accurate method for predicting NAT for breast cancer, among which the application of MRI is the most common. However, due to subjective differences in manual film reading and a large amount of film reading work, etc., the accuracy and efficiency of manual diagnosis of MRI are not ideal. At the same time, the deep image features of MRI can more accurately predict the efficacy of NAT, which cannot be recognized and obtained by the naked eye. Through artificial intelligence machine learning, the problems caused by manual errors can be avoided, and the efficacy of NAT treatment for HER2-positive breast cancer can be predicted more accurately and efficiently. Summary of the Invention

[0003] To solve the technical problem of the disconnection between traditional breast imaging examinations and potential pathophysiology in the prior art, the present application provides an HER2-positive breast cancer auxiliary prediction method based on multi-modal data fusion, including the following steps: case screening, selecting eligible HER2-positive breast cancer patients based on inclusion and exclusion criteria; multi-modal data acquisition, obtaining the preoperative pathological evaluation results of the patients and outputting clinical feature data; radiomics analysis, performing lesion segmentation on magnetic resonance images, extracting traditional radiomics features, Hessian matrix-derived features, and topological omics features to obtain combined radiomics features; feature optimization, extracting the above feature data through intra-class correlation coefficient screening, Pearson correlation filtering, and genetic algorithm iteration to obtain an optimal feature data subset; prediction model construction, fusing the combined radiomics features and clinical feature data according to the multi-factor logistic regression algorithm to generate a PCR probability prediction model; biological verification, performing correlation analysis on the combined radiomics features and RNA transcriptome data, and revealing the biological mechanism of the prediction model through gene pathway enrichment.

[0004] In one embodiment, the multi-modal data acquisition step further includes: collecting clinical parameters such as the age, BMI, ER / PR expression status of the patients, synchronously obtaining the gene sequencing data of tumor tissues, and scanning using a standardized DCE-MRI sequence to obtain the clinical feature data.

[0005] In one embodiment, in the radiomics analysis step: a 3D U-Net network is used for automatic lesion segmentation; the traditional radiomics features include first-order statistics, shape features, and texture features; according to the Hessian matrix, derivative feature data is derived, and the feature data includes Laplacian eigenvalues and Gaussian curvatures. In one embodiment, it further includes: obtaining RNA-seq data from the ISPY2 database, performing differential expression gene analysis using the limma package; performing KEGG pathway enrichment analysis on the differential genes, and screening significant pathways with FDR < 0.05; calculating the tumor immune infiltration score using the CIBERSORT algorithm and establishing a correlation model between the radiomics features and the infiltration levels of CD8+ T cells and M2 macrophages.

[0006] In one embodiment, the genetic algorithm optimization is specifically: the initial population size is set to 100 feature subsets; the tournament selection strategy is adopted to retain the top 10% of individuals with high AUC; the mutation probability is set to 0.2, and the crossover probability is 0.7; the termination condition is that the AUC improvement is < 0.01 for 10 consecutive generations.

[0007] In one embodiment, the input variables of the multi-factor logistic regression model include the radiomics feature score and the tumor size, clinical stage, and ER / PR status in the clinical features;

[0008] The model output is the probability of pCR occurrence in the range of 0-1, and the maximum Youden index is achieved when the probability threshold is set to 0.62.

[0009] In one embodiment, the lesion segmentation is performed by multi-planar three-dimensional delineation in the transverse, coronal, and sagittal planes using ITK-SNAP software; the traditional radiomics features include shape features, first-order statistics, gray-level co-occurrence matrix texture features, and wavelet transform features; the Hessian matrix-derived features are calculated through the scipy.ndimage module and include the statistics of vascularity, sphericity, and structural feature values after multi-scale Gaussian smoothing; the topological omics features include Gaussian curvature, mean curvature, principal curvature extreme values, and shape index calculated through the topology_radiomics library.

[0010] The present invention also provides an embodiment, a computer-readable storage medium storing a computer program, and when the program is executed by a processor, the steps of the method according to any one of claims 1-7 are implemented.

[0011] The present invention also provides an embodiment, including a medical image analysis system for an HER2-positive breast cancer auxiliary prediction method based on multi-modal data fusion, a case screening module for selecting eligible HER2-positive breast cancer patients based on inclusion and exclusion criteria; a multi-modal data acquisition module for obtaining the preoperative pathological evaluation results of patients and outputting clinical feature data; a radiomics analysis module for performing lesion segmentation on magnetic resonance images and extracting traditional radiomics features, Hessian matrix-derived features, and topological omics features to obtain combined radiomics features; a feature optimization module for extracting the above feature data through intra-class correlation coefficient screening, Pearson correlation filtering, and genetic algorithm iteration to obtain an optimal feature data subset; a prediction model construction module for fusing the combined radiomics features and clinical feature data according to the multi-factor logistic regression algorithm to generate a PCR probability prediction model; a biological verification module for performing correlation analysis on the combined radiomics features and RNA transcriptome data and revealing the biological mechanism of the prediction model through gene pathway enrichment.

[0012] In one of the embodiments, the prediction model construction module includes: a real-time verification unit: detecting that the slice thickness of the input MRI image is ≤ 3 mm and the signal-to-noise ratio is ≥ 20; a confidence calibration unit: calibrating the model output probability using the Platt scaling method; a version management unit: recording the model iteration version and the corresponding feature weight matrix.

[0013] Beneficial effects

[0014] By integrating preoperative MRI radiomics features, clinicopathological data, and RNA transcriptome data, the limitations of traditional single-modal analysis are broken through. The MRI images adopt strict quality control standards with a slice thickness of ≤ 3 mm and a signal-to-noise ratio of ≥ 20, combined with the extraction of multi-scale Hessian matrix features (vascularity, sphericity features) and topological persistent homology curvature features, significantly improving the ability to characterize tumor heterogeneity. After three-level feature screening (ICC > 0.75 screening, Pearson correlation filtering, and genetic algorithm optimization), ≤ 20 key features are retained, effectively eliminating the interference of redundant information. The final constructed multi-factor logistic model has an AUC of 0.85 (95% CI: 0.79 - 0.91), a sensitivity of 85%, and a specificity of 82%, which is more than 20% higher than the traditional clinical assessment method (AUC ≈ 0.65 - 0.70), significantly reducing the false negative risk. Through GSEA enrichment analysis, the significant association between key radiomics features and the PI3K-AKT signaling pathway (r ≤ -0.4, p < 0.01), T cell activation pathway (positively correlated with the CD8+ T cell infiltration score) is confirmed, revealing the treatment response mechanism at the molecular level. For example, the vascularity feature is strongly correlated with VEGFR2 mRNA expression (r ≥ 0.5, p < 0.005), providing target clues for anti-angiogenic therapy. The feature contribution heat map and interactive report (including probability cloud map, three-dimensional feature localization) output by the model enable doctors to intuitively understand the prediction basis, and the clinical decision acceptance rate is increased by more than 40%. The real-time analysis engine can complete the generation of prediction results from image segmentation within 5 minutes; the visualization terminal provides functions such as dynamic threshold adjustment and feature-gene linkage analysis. After integration with the PACS system, the clinical use efficiency is increased by more than 3 times. In addition, by standardizing the operation process in the form of a kit (including MRI scan parameter guidance, feature extraction SOP), the result comparability between different medical institutions reaches more than 90%, effectively solving the pain point of low repeatability in existing radiomics research. When the prediction probability threshold is set to 0.63, patients sensitive (high pCR probability) and resistant (high non-pCR probability) to NAT treatment can be accurately identified. Adjusting the treatment plan in advance for resistant patients (such as combining immunotherapy) can increase the pathological complete remission rate by 15% - 20%, avoiding the toxicity accumulation caused by ineffective treatment. At the same time, the association model between immune score and imaging features provides a quantitative basis for formulating individualized treatment strategies (such as evaluating the applicability of PD-1 inhibitors), promoting the practice of precision medicine.

[0015] In summary, through the full-chain innovation of "multimodal data fusion - interpretable model construction - systematic implementation and application", this solution has overcome the three major problems of insufficient accuracy, unclear mechanism, and low clinical practicability in the prediction of the efficacy of neoadjuvant breast cancer treatment, providing reliable technical support for personalized cancer treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0017] Figure 1 It is a working step diagram of a HER2-positive breast cancer auxiliary prediction method based on multimodal data fusion provided by the first embodiment of the present invention;

[0018] Figure 2 It is an AUC diagram of a multi-dimensional radiomics model provided by the first embodiment of the present invention;

[0019] Figure 3 It is a GSEA pathway enrichment diagram provided by the first embodiment of the present invention;

[0020] Figure 4 It is a block diagram of a medical image analysis system provided by the second embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0022] It should be noted that if there are directional indications (such as up, down, left, right, front, back...) involved in the embodiments of the present invention, the directional indications are only used to explain the relative positional relationship and movement conditions between components in a specific posture. If the specific posture changes, the directional indications will also change accordingly.

[0023] In addition, if the embodiments of the present invention involve descriptions such as "first" and "second", the descriptions of "first", "second", etc. are for descriptive purposes only, and should not be construed as indicating or implying their relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, if "and / or" or "and / or" appears throughout the text, its meaning includes three parallel scenarios. Taking "A and / or B" as an example, it includes scenario A, or scenario B, or the scenario where both A and B are satisfied simultaneously. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0024] Example 1

[0025] Breast cancer is one of the most common malignant tumors in women globally. HER2-positive breast cancer is characterized by strong invasiveness and poor prognosis. Neoadjuvant therapy plays an important role in the treatment of HER2-positive breast cancer, which can downstage the tumor and improve the surgical resection rate and breast conservation rate. However, how to accurately predict the efficacy of neoadjuvant therapy has always been a challenge faced in clinical practice. The present invention provides an interpretable method for early predicting the efficacy of neoadjuvant therapy for HER2-positive breast cancer. By integrating multimodal data and advanced data analysis techniques, it realizes the accurate prediction of the treatment efficacy and reveals its internal connection from the biological mechanism level, providing strong support for clinical treatment decisions.

[0026] Reference Figures 1 - 3 , the present application provides an auxiliary prediction method for HER2-positive breast cancer based on multimodal data fusion

[0027] It includes the following steps: case screening, selecting HER2-positive breast cancer patients meeting the requirements based on inclusion criteria and exclusion criteria; multimodal data collection, obtaining the preoperative pathological evaluation results of the patients; radiomics analysis, segmenting lesions in magnetic resonance images, and extracting traditional radiomics features, Hessian matrix-derived features, and topological omics features, where the topological omics features are calculated based on persistent homology analysis; feature optimization, obtaining the optimal subset through intraclass correlation coefficient screening, Pearson correlation filtering, and genetic algorithm iteration; prediction model construction, using a multifactorial logistic regression algorithm to fuse radiomics features and clinical features to generate a pCR probability prediction model; biological verification, performing correlation analysis between radiomics features and RNA transcriptome data, and revealing the biological mechanism of the prediction model through gene pathway enrichment.

[0028] First, a patient screening step is required. The following are the inclusion criteria for screening: Age and gender: Specifically limited to female patients, and the age is greater than 18 years old. This is based on the high incidence of breast cancer in the female population, and the relatively mature physical development of women over 18 years old, whose physical conditions and disease characteristics are more in line with the scope of this study. There is no special regulation on the upper age limit to include as wide a patient population as possible. However, during the actual screening process, the overall health status of the patient will be comprehensively considered. Patients with too high an age and severe underlying diseases may be excluded in subsequent evaluations; Disease diagnosis and staging: The patient must have been pathologically diagnosed as HER2-positive breast cancer without distant organ metastasis. Pathological histological diagnosis is the gold standard for the diagnosis of breast cancer. Through a series of pathological examinations such as slicing and staining of tumor tissues, the type, grade, and HER2 expression of the tumor can be accurately judged. The determination of HER2 positivity adopts the standard of immunohistochemistry (IHC) score 3+ or in situ hybridization (ISH) positivity. For the judgment of no distant organ metastasis, whole-body imaging examinations such as PET-CT and bone scan will be comprehensively used to ensure that the patient is in a stage where neoadjuvant therapy and subsequent surgery can be performed; Surgical situation: The patient has undergone tumor resection surgery and axillary lymph node dissection. This condition is set to obtain complete tumor tissues and lymph node samples for subsequent pathological evaluation and data collection. The methods of tumor resection surgery include breast-conserving surgery and mastectomy. The scope of axillary lymph node dissection is determined according to the clinical stage and the specific situation of the patient. Generally, axillary lymph nodes at Level I-II are dissected. Through surgical resection and lymph node dissection, the size, invasion range, and lymph node metastasis of the tumor can be accurately judged, providing an important basis for the evaluation of the efficacy of neoadjuvant therapy; Imaging examination: The patient has undergone preoperative breast magnetic resonance examination. Breast magnetic resonance imaging (MRI) has the advantages of high soft tissue resolution and multi-parameter imaging, and can clearly show the morphology, size, location of breast tumors and their relationship with surrounding tissues, which is of great value for the diagnosis and staging of breast cancer. In this study, preoperative breast MRI examination is used for subsequent radiomics analysis to provide a data basis for extracting tumor-related features.

[0029] Then, exclusion criteria are set. If there are artifacts in the MRI images, the patient will be excluded. Artifacts in MRI images will affect the quality and clarity of the images, resulting in inaccurate delineation of the tumor area, and further affecting the extraction and analysis results of radiomics features. Common MRI image artifacts include motion artifacts, metal artifacts, chemical shift artifacts, etc. For images with artifacts, attempts will first be made to correct them through image processing techniques. If the image quality requirements still cannot be met after correction, the patient will be excluded.

[0030] Patients who have not received a complete neoadjuvant treatment regimen before surgical resection are excluded. A complete neoadjuvant treatment regimen is crucial for accurately evaluating the therapeutic efficacy. If a patient fails to complete the treatment according to the established regimen, it may lead to a deviation in the treatment effect and fail to truly reflect the role of neoadjuvant treatment on tumors. The neoadjuvant treatment regimen usually includes chemotherapy, targeted therapy, etc. The specific regimen is formulated according to the patient's condition and physical status, and the patient needs to complete the corresponding treatment cycles within the specified time.

[0031] Patients pathologically and histologically diagnosed with bilateral breast cancer are not included in the study. The pathogenesis and clinical characteristics of bilateral breast cancer are different from those of unilateral breast cancer. At the same time, including patients with bilateral breast cancer will increase the complexity of the study and is not conducive to the accurate study of the efficacy of neoadjuvant treatment for HER2-positive breast cancer. During the screening process, through detailed medical history inquiry, imaging examinations, and pathological diagnoses, it is accurately determined whether the patient has bilateral breast cancer.

[0032] According to the above inclusion criteria and exclusion criteria, data collection is carried out for cases meeting the inclusion conditions. The collected content mainly includes data such as magnetic resonance images and clinical characteristics. During the data collection process, ethical norms are strictly followed to ensure the protection of patients' privacy and obtain the informed consent of the patients.

[0033] A dedicated data collection team is established, consisting of doctors, nurses, and data administrators. Doctors are responsible for evaluating and diagnosing the patient's condition to determine whether the patient meets the inclusion conditions; nurses assist doctors in collecting and organizing patient information; data administrators are responsible for entering, storing, and managing the collected data. During the data collection process, standardized data collection forms are used to ensure the accuracy and integrity of the data.

[0034] To ensure data quality, a series of quality control measures are taken. Before data entry, data collectors are trained to familiarize them with data collection standards and procedures. After data entry, data audits are carried out, including checks for data consistency, logic, and integrity. For data in doubt, timely communication and verification are carried out with relevant personnel. At the same time, data is regularly backed up to prevent data loss.

[0035] Collection and processing of patient sample data, including the acquisition of magnetic resonance images, and collecting data on breast MR imaging of patients using 1.5T or 3.0T field strength magnets. These two field strength magnetic resonance devices are widely used in clinical practice and can provide high-quality breast images. The 1.5T field strength magnetic resonance device has a high cost performance, and the image quality can meet most clinical needs; the 3.0T field strength magnetic resonance device has a higher signal-to-noise ratio and resolution, and can more clearly show the fine structure of breast tumors. During the actual acquisition process, the appropriate field strength is selected according to the hospital's equipment configuration and the patient's specific situation.

[0036] Breast MR imaging uses a variety of sequences, including T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), diffusion-weighted imaging (DWI), and dynamic contrast-enhanced imaging (DCE-MRI). T1WI is used to display the anatomical structure of the breast, T2WI can prominently show the signal characteristics of breast tumors, DWI can reflect the water molecule diffusion in tumor tissues, and DCE-MRI observes the hemodynamic changes of tumors by injecting contrast agents. These sequences complement each other and provide rich information for a comprehensive assessment of breast tumors.

[0037] During the image acquisition process, the acquisition parameters are strictly controlled to ensure the consistency of image quality. For example, the parameter settings of the T1WI sequence are: repetition time (TR) = 600 - 800 ms, echo time (TE) = 10 - 20 ms, slice thickness = 3 - 5 mm; the parameter settings of the T2WI sequence are: TR = 3000 - 5000 ms, TE = 80 - 120 ms, slice thickness = 3 - 5 mm; the parameter settings of the DWI sequence are: TR = 4000 - 6000 ms, TE = 60

[0038] - 80 ms, and the b value is selected as 0, 1000 s / mm 2 ; the parameter settings of the DCE-MRI sequence are: TR = 4 - 6 ms, TE = 1 - 3 ms, flip angle (FA) = 10 - 20°, the contrast agent is gadopentetate dimeglumine, the dose is 0.1 mmol / kg, the injection rate is 2 - 3 ml / s, and then 20 ml of normal saline is used for flushing.

[0039] In addition to collecting the magnetic resonance image data of the patients in this hospital, relevant data are also obtained from the TCIA-ACRIN 6698 database. This database contains a large amount of breast cancer imaging data. By integrating with the data of this hospital, the sample size can be expanded and the reliability of the research results can be improved. When using the data in the database, the usage regulations of the database are strictly followed to ensure the legal use of the data.

[0040] Basis for pathological evaluation: According to the Miller&Payne histological grading system, pathologists conduct pathological evaluations on the surgical specimens of patients who have completed neoadjuvant therapy (NAT). This grading system mainly grades according to the proportion of tumor cells reduced after treatment and the morphology of residual tumor cells, divided into grades G1 - G5. Grade G1 indicates no obvious change in tumor cells; Grade G2 indicates a slight reduction in tumor cells; Grade G3 indicates a moderate reduction in tumor cells; Grade G4 indicates a significant reduction in tumor cells; Grade G5 indicates the complete disappearance of tumor cells, that is, pathological complete remission (pCR). Grades G1 - G4 are classified into the non-pathological complete remission (non-pCR) group, and G5 is classified into the pathological complete remission (pCR) group. This grouping method can clearly distinguish the efficacy of neoadjuvant therapy and provide clear labels for subsequent data analysis and model construction. During the pathological evaluation process, pathologists will observe multiple sections of the surgical specimens to ensure the accuracy of the evaluation results. At the same time, to improve the consistency of the evaluation, pathologists participating in the evaluation are uniformly trained to familiarize them with the standards and operation procedures of the Miller&Payne grading system.

[0041] Next, the working steps for clinical data collection are carried out. The content of clinical feature information collection is rich and diverse, including age at diagnosis, number of tumors, tumor size, pathological type, clinical tumor stage, clinical lymph node stage, ER status, PR status, molecular subtype, surgical method, and chemotherapy regimen, etc. The age at diagnosis reflects the age characteristics of patients at the time of onset and has certain reference value for studying the pathogenesis and prognosis of breast cancer. The number and size of tumors are directly related to the tumor burden and the severity of the disease. Common pathological types include invasive ductal carcinoma, invasive lobular carcinoma, etc. Breast cancers of different pathological types have differences in biological behavior and treatment response. The clinical tumor stage and lymph node stage are judged according to the TNM staging system of the International Union Against Cancer (UICC) and are used to evaluate the degree of tumor progression. The ER (estrogen receptor) and PR (progesterone receptor) statuses have important guiding significance for the selection of endocrine therapy. The molecular subtype is divided into Luminal A type, Luminal B type, HER2 overexpression type, and triple-negative type according to the expression of ER, PR, HER2, and Ki-67. The treatment strategies and prognoses of breast cancers of different molecular subtypes are different. The surgical methods include breast-conserving surgery and mastectomy, and the chemotherapy regimen is selected according to the specific situation of the patient, such as the anthracycline combined with taxane regimen, etc.

[0042] The collection of clinical data is mainly carried out by consulting the medical records of patients, communicating with clinicians, and conducting patient questionnaires. When consulting medical records, carefully check all information to ensure the accuracy of the data. For some uncertain or missing data, promptly verify and supplement it with clinicians. Patient questionnaires are mainly used to collect some subjective information, such as the patient's living habits, family history, etc., which may affect the incidence and treatment of breast cancer. Organize the collected clinical data and enter it into a spreadsheet or database in a unified format. During the entry process, standardize the data. For example, unify the unit of tumor size to centimeters, and unify the expression results of ER and PR status to positive or negative, etc. At the same time, conduct logical verification on the data to ensure the consistency and integrity of the data. For example, check whether the clinical tumor stage and lymph node stage are consistent with other relevant information. If contradictory data is found, promptly verify and correct it.

[0043] Use the ITK-SNAP software to observe the cross-sectional, coronal, and sagittal planes of the MRI images, and then layer by layer outline the tumor area. The ITK-SNAP software has the characteristics of simple operation and powerful functions, and can provide a variety of image segmentation tools and visualization functions, which is convenient for radiologists to accurately outline the tumor area. When using the software, radiologists can flexibly select different segmentation tools according to the shape and signal characteristics of the tumor, such as manual segmentation, semi-automatic segmentation, etc. Radiologists first start from the top layer of the tumor on the cross-sectional image of the MRI image and outline along the tumor boundary to ensure that the tumor tissue is completely included in the outlined area. During the outlining process, pay attention to distinguishing the tumor from the surrounding normal tissues. For some areas with blurred boundaries, combine the coronal and sagittal plane images for comprehensive judgment. After completing the outlining of the cross-sectional image, outline the tumor area of the coronal and sagittal plane images in the same way. Finally, check and correct the outlined tumor area to ensure the accuracy and integrity of the outlining. To ensure the accuracy of the image segmentation results, a double-check method is adopted. That is, another experienced radiologist reviews the results outlined by the first doctor. If any inconsistencies are found, the two doctors discuss together and determine based on the image characteristics and clinical information. At the same time, regularly evaluate the quality of the segmentation results, calculate the consistency index of the segmentation results, such as the Dice coefficient, etc. If it is found that the segmentation results have a large deviation, promptly retrain and guide the radiologists.

[0044] The pyradiomics software was used to extract the radiomics features from the data in this experiment. This software is an open-source software specifically for radiomics feature extraction, with rich feature extraction algorithms and functions. Before using the pyradiomics software, install and configure it to ensure that the software can run properly. At the same time, learn and train on the usage method of the software to be familiar with the principles and parameter settings of various feature extraction algorithms.

[0045] Traditional radiomics feature extraction: Traditional omics features include shape features, first-order statistical features (intensity features), texture features, and higher-order features based on transformations. Shape features mainly reflect the morphological information of tumors, such as volume, surface area, sphericity, compactness, etc. The volume is obtained by calculating the number of voxels within the delineated region; the surface area is calculated by computing the surface of the delineated region; sphericity is used to measure the closeness of the tumor shape to a sphere, and the calculation formula is: \(4\pi(3V / 4\pi)^{2 / 3} / S\), where V is the volume and S is the surface area; compactness reflects the compactness of the tumor shape, and the calculation formula is: \(S^3 / (36\pi V^2)\). First-order statistical features mainly describe the gray-scale distribution of the image, such as mean, standard deviation, skewness, kurtosis, etc. The mean represents the average gray-scale of the image, the standard deviation reflects the dispersion degree of the image gray-scale, skewness is used to measure the symmetry of the image gray-scale distribution, and kurtosis reflects the steepness of the image gray-scale distribution. Texture features are extracted by analyzing the spatial distribution relationship of pixel gray-scales in the image. Common texture feature extraction methods include gray-level co-occurrence matrix (GLCM), gray-level run-length matrix (GLRLM), etc. GLCM features include contrast, correlation, energy, entropy, etc., and are obtained by calculating the gray-level co-occurrence relationship between pixel pairs in different directions and at different distances; GLRLM features include short-run emphasis, long-run emphasis, gray-level non-uniformity, run-length non-uniformity, etc., and are obtained by analyzing the run-length distribution of pixels with the same gray-level value in the image. Higher-order features based on transformations are extracted after performing mathematical transformations on the image, such as high-frequency coefficient features after wavelet transformation.

[0046] Hessian Matrix Feature Extraction: The features of the Hessian matrix are calculated using the scipy.ndimage module. First, the image is smoothed with Gaussian filters at different scales, and the scale parameter of the Gaussian smoothing is adjusted according to the resolution and noise level of the image. Gaussian smoothing can reduce the impact of image noise on feature extraction while highlighting the structural information of the image. Then, the eigenvalues of the Hessian matrix are calculated point by point within the region of interest. The Hessian matrix is a second-order partial derivative matrix. For a two-dimensional image, the structural features reflect the complexity of the internal structure of the tumor, the shape index is used to describe the shape features of the tumor, and the curvature includes the mean curvature, Gaussian curvature, etc., which can reflect the degree of curvature of the tumor surface. For each feature, its statistics (minimum value, maximum value, mean value, standard deviation, and quantiles) are further calculated to characterize the overall distribution of the feature. For example, calculating the mean value of the vascular features can reflect the overall vascular distribution level of the tumor, and the standard deviation can reflect the dispersion degree of the vascular distribution.

[0047] Topological omics features are analyzed for the region of interest through the topology_radiomics library. The mask data is preprocessed into a single lesion region, that is, the background information unrelated to the tumor in the image is removed, and only the tumor region is retained. Then, its curvature and morphological features are calculated, including Gaussian curvature, mean curvature, principal curvatures (minimum and maximum values), total curvature, curvature property, sharpness, and shape index, etc. Gaussian curvature reflects the degree of curvature of the surface at a certain point, the mean curvature is an important parameter to describe the local shape of the surface, the principal curvatures respectively represent the maximum and minimum degrees of curvature of the surface at a certain point, and the total curvature is the integral of the Gaussian curvature over the entire surface. The curvature property is used to measure the bending characteristics of the surface, and the sharpness reflects the sharpness of the surface edge.

[0048] The present invention also provides an embodiment, a computer-readable storage medium storing a computer program, and when the program is executed by a processor, the steps of any one of the above methods are implemented.

[0049] Example 2

[0050] Reference Figure 4, the present invention also provides an embodiment, including a medical image analysis system for an HER2-positive breast cancer auxiliary prediction method based on multimodal data fusion, a case screening module for selecting eligible HER2-positive breast cancer patients based on inclusion criteria and exclusion criteria; a multimodal data acquisition module for obtaining the preoperative pathological evaluation results of patients and outputting clinical feature data; an imaging genomics analysis module for segmenting lesions in magnetic resonance images, extracting traditional imaging genomics features, Hessian matrix-derived features and topological genomics features to obtain combined imaging genomics features; a feature optimization module for extracting the above feature data through intraclass correlation coefficient screening, Pearson correlation filtering and genetic algorithm iteration to obtain an optimal feature data subset; a prediction model construction module for fusing the combined imaging genomics features and clinical feature data according to the multi-factor logistic regression algorithm and generating a PCR probability prediction model in combination; a biological verification module for performing correlation analysis on the combined imaging genomics features and RNA transcriptome data and revealing the biological mechanism of the prediction model through gene pathway enrichment.

[0051] Among them, the prediction model construction module includes: a real-time verification unit: detecting that the slice thickness of the input MRI image ≤ 3 mm and the signal-to-noise ratio ≥ 20; a confidence calibration unit: calibrating the model output probability by using the Platt scaling method; a version management unit: recording the model iteration version and the corresponding feature weight matrix.

[0052] In summary, the present invention provides an HER2-positive breast cancer auxiliary prediction method based on multimodal data fusion, which is used to improve the early prediction of the efficacy of neoadjuvant therapy for HER2-positive breast cancer, and its beneficial effects are significant. In the patient screening link, clear inclusion and exclusion criteria ensure the homogeneity of the research objects, making the results more targeted and reliable. The multimodal data collection covers magnetic resonance images, clinical features and pathological evaluations, providing rich information for comprehensive analysis. The image segmentation is completed by experienced doctors with the help of professional software, and the double-check ensures the accuracy of the segmentation, laying a solid foundation for subsequent feature extraction. And the feature subset is further optimized according to feature screening to improve the model performance. The constructed multi-factor logistic regression prediction model combines imaging genomics and clinical features, and can effectively predict the pCR probability. The biological verification correlates imaging genomics and RNA transcriptome data, reveals the potential biological mechanism, and enhances the interpretability of the model. Generally speaking, this scheme provides a precise and non-invasive prediction tool for clinicians, helps to formulate personalized treatment strategies, and improves the treatment effect and the quality of life of patients.

[0053] The above are only the preferred embodiments of the present invention, and do not thereby limit the patent scope of the present invention. Any equivalent structural transformation made under the inventive concept of the present invention by using the content of the specification and drawings of the present invention, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present invention.

Claims

1. An HER2-positive breast cancer auxiliary prediction method based on multimodal data fusion, characterized in that, It includes the following steps: Case screening, selecting eligible HER2-positive breast cancer patients based on inclusion and exclusion criteria; Multi-modal data collection, obtaining the preoperative pathological evaluation results of patients and outputting clinical feature data; Radiomics analysis, segmenting lesions in magnetic resonance images, extracting traditional radiomics features, Hessian matrix-derived features, and topological omics features, and obtaining combined radiomics features; Feature optimization, extracting the above feature data through intra-class correlation coefficient screening, Pearson correlation filtering, and genetic algorithm iteration to obtain an optimal feature data subset; Prediction model construction, fusing the combined radiomics features and clinical feature data according to the multi-factor logistic regression algorithm, and combining to generate a pCR probability prediction model; Biological verification, performing correlation analysis on the combined radiomics features and RNA transcriptome data, and revealing the biological mechanism of the prediction model through gene pathway enrichment.

2. The HER2-positive breast cancer auxiliary prediction method based on multi-modal data fusion according to claim 1, characterized in that, The multi-modal data collection step further includes: Collecting clinical parameters such as the age, BMI, ER / PR expression status of patients, synchronously obtaining gene sequencing data of tumor tissues, and scanning with a standardized DCE-MRI sequence to obtain the clinical feature data.

3. The HER2-positive breast cancer auxiliary prediction method based on multi-modal data fusion according to claim 1, wherein In the radiomics analysis step: Using a 3D U-Net network for automatic lesion segmentation and extracting traditional radiomics features. The traditional radiomics features include first-order statistics, shape features, and texture features. Then, Hessian matrix-derived features are extracted. The Hessian matrix-derived features are derived from the Hessian matrix, and the feature data includes Laplacian eigenvalues and Gaussian curvature.

4. The HER2-positive breast cancer auxiliary prediction method based on multi-modal data fusion according to claim 1, wherein, It also includes: Obtaining RNA-seq data from the ISPY2 database, performing differential expression gene analysis using the limma package, performing KEGG pathway enrichment analysis on the differential genes, screening significant pathways with FDR < 0.05, calculating the tumor immune infiltration score using the CIBERSORT algorithm, and establishing a correlation model between radiomics features and the infiltration levels of CD8+ T cells and M2 macrophages.

5. The HER2-positive breast cancer auxiliary prediction method based on multimodal data fusion according to claim 1, wherein The specific steps of genetic algorithm optimization are as follows: The initial population size is set to 100 feature subsets; according to the tournament selection strategy, the top 10% of individuals with high AUC are retained; the mutation probability can be set to 0.2, and the crossover probability can be set to 0.

7.

6. The HER2-positive breast cancer auxiliary prediction method based on multi-modal data fusion according to claim 1, wherein The input variables of the multi-factor logistic regression model include the radiomics feature score and the tumor size, clinical stage, and ER / PR status in clinical features. The model output is the pCR occurrence probability in the range of 0-1. When the probability threshold is set to 0.62, the maximum Youden index is achieved.

7. The HER2-positive breast cancer auxiliary prediction method based on multi-modal data fusion according to claim 1, wherein Lesion segmentation was performed by multi-planar three-dimensional delineation on the transverse, coronal, and sagittal planes using ITK-SNAP software; traditional radiomics features included shape features, first-order statistics, gray-level co-occurrence matrix texture features, and wavelet transform features; Hessian matrix-derived features were calculated through the scipy.ndimage module, including vascularity, sphericity, and structural eigenvalue statistics after multi-scale Gaussian smoothing; the topological omics features included Gaussian curvature, mean curvature, principal curvature extrema, and shape index calculated through the topology_radiomics library.

8. A computer-readable storage medium storing a computer program, characterized in that, When the program is executed by a processor, it implements the steps of the HER2-positive breast cancer auxiliary prediction method based on multi-modal data fusion according to any one of claims 1-7.

9. A medical image analysis system, characterized in that, The uses include: A case screening module for selecting eligible HER2-positive breast cancer patients based on inclusion and exclusion criteria; A multi-modal data acquisition module for obtaining the preoperative pathological evaluation results of patients and outputting clinical feature data; A radiomics analysis module for performing lesion segmentation on magnetic resonance images, extracting traditional radiomics features, Hessian matrix-derived features, and topological omics features, and obtaining combined radiomics features; A feature optimization module for extracting the above feature data through intra-class correlation coefficient screening, Pearson correlation filtering, and genetic algorithm iteration to obtain an optimal feature data subset; A prediction model construction module for fusing the combined radiomics features and clinical feature data according to the multi-factor logistic regression algorithm and generating a PCR probability prediction model; A biological verification module for performing correlation analysis between the combined radiomics features and RNA transcriptome data and revealing the biological mechanism of the prediction model through gene pathway enrichment.

10. The medical image analysis system according to claim 9, wherein The prediction model construction module includes: A real-time verification unit for detecting the slice thickness of the input MRI image with a preset signal-to-noise ratio ≥ 20; A confidence calibration unit for calibrating the model output probability using the Platt scaling method; A version management unit for recording the model iteration version and the corresponding feature weight matrix.

Images