Predictive diagnostic system based on medical images and digital features of breast cancer

Abstract

This invention discloses a predictive diagnostic system for breast cancer based on medical images and digital features. The system collects breast cancer-related physical examination digital features and breast tissue pathology image data. After data preprocessing, it constructs prediction models based on machine learning (such as decision trees, random forests, and support vector machines) and deep learning (such as CNN and Transformer-based network architectures). A soft-voting ensemble learning strategy is then used to fuse the prediction results of each model, thereby achieving early diagnosis of breast cancer. This solution effectively utilizes the advantages of multimodal data and ensures prediction accuracy and robustness through model selection and ensemble, providing auxiliary diagnostic evidence for clinical practice and reducing the risk of misdiagnosis.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and artificial intelligence, specifically to a deep learning and machine learning prediction model based on breast cancer medical images and conventional digital detection features, and a system for early breast cancer diagnosis using ensemble learning. Background Technology

[0002] Breast cancer is typically classified into stages 0 to 4 based on the proliferation of cancer cells. Stages 0 and 1 are considered early stages. Recent clinical findings indicate that the average 10-year survival rate for breast cancer has reached 60%, while the survival rate after treatment for early-stage (stage 1) breast cancer can reach 80%, and the survival rate for stage 0 breast cancer is even close to 100%. Cancer statistics reports show that the relative survival rate (5-year survival percentage compared to the survival rate of the same-sex age-appropriate normal population) of patients diagnosed with early-stage breast cancer is significantly higher than that of patients diagnosed with advanced-stage breast cancer. Specifically, the 5-year survival rate for patients diagnosed with stage 0 and 1 breast cancer is close to 100%, for stage 2 it is 93%, while for stage 3 it drops to 72%.

[0003] Therefore, early detection, early diagnosis, and early treatment are key to reducing breast cancer mortality and improving prognosis. Summary of the Invention

[0004] Based on this, one objective of the present invention is to provide a predictive diagnostic system for breast cancer based on medical images and digital features. This system utilizes deep learning and machine learning algorithms to extract and classify features from breast tissue slice images and routine biochemical test data, respectively. Through ensemble learning, it achieves high accuracy, stability, and robustness in early prediction and auxiliary diagnosis of breast cancer, thereby significantly improving the accuracy of early detection of breast cancer and providing a reliable basis for clinical diagnosis.

[0005] In a first aspect, the present invention provides a predictive diagnostic system for breast cancer based on medical images and digital features. When the system is in use, after determining breast cancer using a digital feature module, it further utilizes an image feature module and a soft voting strategy execution module to determine whether the breast cancer result is benign or malignant. Specifically: the digital feature module is configured to include K1 machine learning models, each learning model performs breast cancer classification learning based on physical examination digital feature samples, retaining the optimal machine learning model. Each physical examination digital feature sample includes age, BMI, glucose, insulin, HOMA, leptin, adiponectin, resistin, and MCP-1. The image feature module is configured to include K2 deep network models, each deep network model taking breast tissue pathology images as input and breast cancer probability as output. Each deep network model is trained based on a breast tissue pathology image sample set, and N optimal deep network models are selected, where N is less than K2. The soft voting strategy execution module is configured to fuse the output probabilities of the N optimal deep network models using soft voting, fusing the features. , Let be the probability of the i-th model. The weighted values ​​of the model are used to determine the breast cancer outcome based on the fusion features.

[0006] In a second aspect, the present invention provides a predictive diagnostic device for breast cancer based on medical images and digital features, characterized in that it includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed by any of the systems disclosed herein.

[0007] The beneficial technical effects of this invention are as follows: Utilizing multimodal data for breast cancer diagnosis, the system employs multi-model selection and ensemble learning during diagnosis, effectively reducing the bias and limitations of a single model. This achieves highly accurate and robust early diagnosis of breast cancer, providing visual evidence and offering advantages for easy promotion and clinical decision support. Furthermore, the system can adaptively update the integrated model based on current model developments, ensuring the accuracy and robustness of breast cancer diagnosis. Attached Figure Description

[0008] Figure 1 Example chart showing statistical parameters for breast cancer screening data.

[0009] Figure 2 This is an example of a correlation heatmap generated based on breast cancer screening data.

[0010] Figure 3 This is a schematic diagram of the decision-making process in a decision tree model.

[0011] Figure 4 This is a schematic diagram of a deep learning network structure (taking ResNet18 as an example).

[0012] Figure 5-1 This chart compares the average accuracy of different deep learning models.

[0013] Figure 5-2 A chart comparing the standard deviations of different deep learning models.

[0014] Figure 5-3 This is a comparison chart of training times for different deep learning models.

[0015] Figure 6 This is a schematic diagram of the Voting integrated learning framework structure.

[0016] Figure 7-1 This document demonstrates the process and examples for generating saliency plots for benign samples.

[0017] Figure 7-2 This document demonstrates the process and examples for generating saliency maps for malignant samples.

[0018] Figure 8 This is a schematic diagram of the system diagnostic process. Detailed Implementation

[0019] Given the lack of effective primary prevention strategies for breast cancer and the crucial role of early prediction in improving patient survival rates, this study established an early breast cancer prediction model based on decision trees, random forests, and SVM (Support Vector Machine). This model uses age, BMI, glucose, insulin, and adiponectin as statistical features. Simultaneously, early breast tissue slice identification models based on CNN and Transformer architectures were also developed, aiming to provide a scientific basis for the early diagnosis and treatment of breast cancer.

[0020] The following examples further illustrate how the present invention can be implemented. It is clear and complete that the described embodiments are merely a part of the embodiments described in this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments described in this application without inventive effort are within the scope of protection of this application.

[0021] Example 1: Determining whether someone has breast cancer Data Preparation and Preprocessing: 116 data samples from the UCI database were used as the digital characteristic sample set for physical examinations. Each sample included age, BMI, glucose, insulin, HOMA, leptin, adiponectin, resistin, and MCP-1. See statistical examples below. Figure 1 .

[0022] Descriptive statistics, correlation analysis, and Kolmogorov-Smirnov tests were performed on the collected data samples, and a heatmap was generated (see [link to heatmap]). Figure 2 Paired feature graphs are used for visualization analysis.

[0023] Digital Feature Module: Constructs K1 machine learning models, each classifying breast cancer based on physical examination digital feature samples. The classification result is either healthy or diseased. Machine learning models include decision trees, random forests, and support vector machines.

[0024] Specifically, a decision tree model is constructed, entropy is selected as the splitting criterion, and reasonable maximum depth and minimum number of leaf node samples are set, such as a maximum depth of 3 and a minimum number of leaf nodes of 5.

[0025] To construct a random forest model, the construction method can adopt bootstrap sampling and random feature selection, and set key parameters such as n_estimators, max_depth, min_samples_leaf, and max_features.

[0026] Construct a Support Vector Machine (SVM) model, set the kernel to "linear", and set appropriate penalty parameters C=5 and kernel parameters y=1.

[0027] Figure 3 This diagram illustrates the decision-making process using a decision tree model based on 116 data samples from the UCI database.

[0028] Parameter Optimization and Evaluation: The hyperparameters of the constructed machine learning models were optimized using a grid search method, and parameter sets with better AUC (Area Under Curve) were selected. Cross-validation was used to evaluate the performance of the optimized models, and ROC curves, accuracy (ACC), F1 score, F2 score, sensitivity (SEN), specificity (SPEC), and negative predictive value (NPV) were output for auxiliary judgment. If the AUC is good, but a certain auxiliary judgment metric is poor, it indicates that the machine learning model has a reliability problem, and the model needs to be investigated and analyzed.

[0029] The results show that the digital feature model can make preliminary classification and prediction of the early risk of breast cancer, among which the SVM model optimized by grid search shows high predictive ability on multiple indicators.

[0030] Example 2: Determining the Type of Breast Cancer When the digital feature module determines that a patient has breast cancer, the image feature module and the soft voting strategy execution module are further used to determine the type of breast cancer. The type determination includes whether it is benign or malignant. Furthermore, the stage of malignancy can also be determined.

[0031] Data preprocessing module: This module performs size unification, normalization, and data augmentation on 7909 breast tissue pathology images from the BreakHis dataset to construct the input dataset for the deep learning model. See Table 1 for the BreakHis dataset structure diagram.

[0032] Table 1

[0033] Image feature module (base learner): Constructs K2 deep network models. One construction method uses CNN-based models, including lightweight and deep networks such as ResNet18, MobileNetV2, MobileNetV3, EfficientNet, RegNet, ShuffleNet, and ConvNeXt. Simultaneously, it constructs Transformer-based models, including ViT and TNT, to segment the image and extract global features using a self-attention mechanism. Figure 4 This is a schematic diagram of the ResNet18 network structure built on CNN.

[0034] Model Training and Evaluation: Each model undergoes a preset number of training rounds (e.g., 21 rounds), and the average accuracy, standard deviation of accuracy, and training time are recorded to compare the performance of each model in image feature extraction and classification tasks. N models are selected for subsequent ensemble. The standard deviation of accuracy measures the stability of the model, and the training time measures the computational speed. In one implementation, N models with high average accuracy, low standard deviation of accuracy, and short training time are selected for ensemble. In another implementation, when the standard deviation of accuracy measures the stability of the model and meets a preset stability threshold, N models with high average accuracy are selected for ensemble, and the required hardware computing equipment is determined based on the training time. Figure 5-1 , Figure 5-2 and Figure 5-3 This is a comparison chart of the average accuracy, standard deviation, and training time of different deep learning models in one implementation method. N is less than K².

[0035] Saliency map generation: The gradient of the image input is calculated using backpropagation to generate a saliency map, as shown in Figure 5. This map is used to verify the model's attention to breast pathological features (such as cell arrangement and edge morphology) and ensure that the model learns effective pathological information. See the example of the saliency map. Figure 7-1 and Figure 7-2 As shown, the process and examples of generating saliency maps for benign and malignant samples are presented respectively.

[0036] Soft voting strategy execution module (strong learner): See the Voting ensemble learning framework structure diagram. Figure 6 The output probabilities of N high-performing deep learning models (such as CNN and Transformer architectures) are fused using a soft voting method to incorporate the results of multiple deep learning models into joint decision-making. The fusion formula is as follows: R represents the fusion feature. Let be the probability of the i-th model. The weights are used to assign weights to the model. The output determines whether breast cancer is benign or malignant based on the fused features. In some implementations, the specific stage of malignancy is output.

[0037] The data samples in the aforementioned digital feature module and image feature module can be centrally preprocessed through the data preprocessing module. In the data preprocessing module, the digital feature samples from physical examinations are normalized, outlier detected, and correlation analyzed; while the breast tissue pathology image data are normalized, sized, augmented, and noise filtered.

[0038] Configure a visualization module in the system to display the ROC curve (i.e., receiver operating characteristic curve), prediction results, and significance plots to assist clinical decision-making.

[0039] Experiments have demonstrated that this diagnostic system first uses a digital feature model to diagnose whether a disease is present. Then, when determining if breast cancer is present, it further utilizes an image feature model and a soft-voting strategy execution module to determine the type of breast cancer. In the early prediction of breast cancer, this system achieves higher accuracy and stability than a single model, providing a powerful auxiliary diagnostic tool for clinical practice. See the diagnostic flowchart for details. Figure 8 .

[0040] In summary, the present invention can be a system and / or a computer program product device. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present invention.

[0041] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0042] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0043] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0044] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A predictive diagnostic system for breast cancer based on medical images and digital features, characterized in that: When the system is in use, after determining whether a patient has breast cancer using the digital feature module, it further utilizes the image feature module and the soft voting strategy execution module to determine whether the breast cancer is benign or malignant; wherein: The digital feature module is configured to include K1 machine learning models. Each machine learning model learns breast cancer classification based on physical examination digital feature samples. The optimal machine learning model is retained. Each physical examination digital feature sample includes age, BMI, glucose, insulin, HOMA, leptin, adiponectin, resistin, and MCP-1. The image feature module is configured to include K2 deep network models, each of which takes a breast tissue pathology image as input and outputs the probability of breast cancer. Each deep network model is trained based on a breast tissue pathology image sample set, and N deep network models with the best performance are selected, where N is less than K2. The soft voting strategy execution module is configured to fuse the output probabilities of N best-performing deep network models using soft voting, and fuse the features. , Let be the probability of the i-th model. To assign weights to the model, the fusion features are used to determine whether breast cancer is benign or malignant.

2. The prediction system according to claim 1, characterized in that, Machine learning models include decision trees, random forests, and support vector machines.

3. The prediction system according to claim 1, characterized in that, Deep network models include deep learning models based on convolutional neural networks (CNNs) and those based on the Transformer architecture.

4. The prediction system according to claim 1, characterized in that, The samples underwent preprocessing, including normalization, outlier detection, and correlation analysis of digital feature samples from physical examinations, as well as normalization, size unification, data enhancement, and noise filtering of breast tissue pathology image data.

5. The prediction system according to claim 1, characterized in that, The prediction system includes a visualization module configured to display ROC curves, prediction results, and a saliency map, which is generated by calculating the gradient of the image input using a backpropagation method.

6. The prediction system according to claim 1, characterized in that, The machine learning model uses grid search or other optimization algorithms to optimize the hyperparameters of the model.

7. The prediction system according to claim 1, characterized in that, Each machine learning model was evaluated using cross-validation.

8. The prediction system according to claim 1, characterized in that, The digital signature module's assessment of breast cancer also includes health status.

9. The prediction system according to claim 1, characterized in that, When the breast cancer result is malignant, further determination of the malignancy stage is needed.

10. A predictive diagnostic device for breast cancer based on medical images and digital features, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed by the system as claimed in any one of claims 1 to 9.

Images