Preferred integrated system and method for enabling real-time governance of continuous deep learning models

Abstract

The application discloses an optimal integrated system and method for realizing real-time supervision of a continuous deep learning model, comprising a continuous learning subsystem, a data processing module, an optimal model generation module, a diagnosis module, a data storage module, a pathological type database, a pathological disease database, an optimal model database and an ultrasonic PACS subsystem, and the realization method comprises the following steps: step one, data set construction; step two, algorithm optimization; step three, system development; and step four, system application.The integrated system is designed by using the model optimization method, realizes supervision of the model, and can guarantee the stability of the diagnosis performance of the integrated system.The integrated system has the continuous learning capability, the diagnosis capability thereof is continuously improved along with stage learning, and the accuracy of the diagnosis result is guaranteed.The integrated system can output multiple diagnosis results by using the model integrated design, and plays a very important auxiliary role in disease identification for doctors.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, specifically to a preferred integrated system and method for realizing real-time monitoring of continuous deep learning models. Background Technology

[0002] Continuous learning holds immense potential in medical practice, similar to the learning methods of human clinicians, where models learn from errors and adjust their performance through gradually increasing data. However, current literature lacks reports on using continuous learning to develop models for ultrasound diagnosis. Existing ultrasound diagnostic models lack continuous learning capabilities, hindering improvements in diagnostic accuracy. Furthermore, due to the impact of AI on health outcomes, continuous learning models must incorporate clinical data from a large number of patients, potentially raising patient privacy concerns. Currently, there are no methods to evaluate the quality of these models. Monitoring AI in real-time medicine presents significant challenges and high risks. Existing technologies lack effective solutions for model monitoring, typically training models more than 20 times until accuracy stops improving, compromising system stability. Finally, to avoid interference between diagnostic results, existing diagnostic models output relatively singular results, failing to provide physicians with sufficient evidence for disease differentiation. Summary of the Invention

[0003] The purpose of this invention is to provide a preferred integrated system and method for realizing real-time monitoring of continuous deep learning models, so as to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: an optimized integrated system for real-time monitoring of a continuous deep learning model, comprising a continuous learning subsystem and an ultrasound PACS subsystem. The continuous learning subsystem includes a data processing module, an optimized model generation module, a diagnostic module, and a data storage module. The ultrasound PACS subsystem is connected to the data processing module, which is connected to the optimized model generation module, the diagnostic module, and the data storage module. Both the optimized model generation module and the diagnostic module are connected to the data storage module, and the diagnostic module is connected to the ultrasound PACS subsystem.

[0005] Preferably, the data processing module includes an image classification submodule and a dataset construction submodule.

[0006] Preferably, the preferred model generation module includes a model training submodule and a model testing submodule, and the model training submodule and the model testing submodule establish a data connection.

[0007] Preferably, the data storage module includes a tumor benign / malignant database, a pathological type database, a pathological disease database, and a preferred model database. The tumor benign / malignant database, the pathological type database, and the pathological disease database are all connected to the data processing module, and the preferred model database is connected to the preferred model generation module and the diagnosis module.

[0008] The implementation method of an optimized ensemble system for real-time monitoring of continuous deep learning models includes the following steps: Step 1, dataset construction; Step 2, algorithm optimization; Step 3, system development; and Step 4, system application.

[0009] In step one above, the dataset construction includes the following steps:

[0010] 1) Constructing the experimental dataset: 499 cases of breast ultrasound with pathological results were selected for training. 965 images of benign breast masses and 965 images of malignant breast masses were collected. 800 images were randomly selected for training and 165 images were selected for testing to construct the experimental dataset (EDS).

[0011] 2) Constructing a test dataset for benign and malignant tumor diagnosis: 130 cases were selected as the test dataset (TDS). Among the cases used for testing, 200 images of benign tumors and 200 images of malignant tumors were selected to construct a test dataset for benign and malignant tumor diagnosis (BMTDS).

[0012] 3) Construct a pathological type diagnostic test dataset: Based on the pathological type of the pathological diagnosis results, 200 images of invasive non-specific carcinoma were selected as the positive class, and 200 images of other pathological types were randomly selected as the negative class to construct a pathological type diagnostic test dataset (PTTDS).

[0013] 4) Construct a pathological disease diagnosis test dataset: Based on the pathological disease diagnosis results, 200 images of invasive ductal carcinoma of the breast were selected as the positive class, and 200 images of other pathological diseases were randomly selected as the negative class to construct a pathological disease diagnosis test dataset (PDTDS).

[0014] In step two above, the algorithm preferably includes the following steps:

[0015] 1) Select the best algorithm: Select 13 commonly used algorithms to develop diagnostic models. Train the model using the EDS obtained in step 1 above and test the model using BMTDS to obtain the AUC value (Area under the curve) of the diagnostic model. Compare the diagnostic performance of the models and select 5 algorithms with better diagnostic performance.

[0016] 2) Selecting the optimal algorithm: Five algorithms were compared with each other, and models were built using each of the five algorithms. BMTDS, PTTDS, and PDTDS were used to test and compare the performance of the algorithms in diagnosing benign and malignant masses, pathological types, and pathological diseases. Among these five algorithms, the algorithm with the highest AUC value was selected as the development algorithm for the system.

[0017] In step three above, based on the dataset constructed in step one and the optimal algorithm obtained in step two, a continuous learning subsystem is developed using Python and integrated with the ultrasound PACS subsystem.

[0018] In step four above, the system application includes the following steps:

[0019] 1) Data input: Physicians provide data and images to the data processing module through the ultrasound PACS subsystem. When the image has pathological results, proceed to step 2); when the image does not have pathological results, proceed to step 3.

[0020] 2) Continuous Learning: When the data processing module receives images with pathological results, it automatically classifies the images and constructs datasets, including a benign and malignant mass dataset (BMS), a pathological type dataset (PTS), and a pathological disease dataset (PDS). When the number of benign and malignant images reaches a preset number, the optimal model generation module automatically starts to train and test the models. Three sets of models are trained using the BMS, PTS, and PDS constructed in step 2). After each set of models is trained, the last 8 models in the set are saved for testing. The model with the highest AUC value is automatically selected as the optimal model in the set. After all three models are trained, the model testing submodule uses the three optimal models to perform a comprehensive test on the TDS and outputs the results.

[0021] 3) Automatic diagnosis: When the data processing module receives an image without pathological results, it transmits the image to the diagnosis module. The diagnosis module first uses BM_OM to perform benign or malignant diagnosis on the mass image. In addition to obtaining benign or malignant diagnosis results, it selects the image with the highest probability of malignancy and performs pathological type diagnosis through PT_OM and pathological disease diagnosis through PD_OM. After completing the diagnosis, it returns the results to the ultrasound PACS subsystem.

[0022] Preferably, in step 1), the 499 data points are divided into 6 training datasets according to the order of the patient's examination, for phased training of the continuous deep learning model.

[0023] Preferably, in step four (2), the specific steps for model training and testing are as follows:

[0024] a) The model training submodule trains and saves benign and malignant diagnostic models through BMS. After training is completed, the last 8 models are selected from the saved models for testing. That is, the model testing submodule is automatically started to test BMTDS, and the model with the highest AUC value is automatically selected as the benign and malignant diagnostic optimization model (BM_OM), and the original records are output and saved.

[0025] b) The model training submodule trains and saves pathological type models through PTS. After training is completed, the last 8 models are selected from the saved models for testing. That is, the model testing submodule is automatically started to test PTTDS, automatically selects the model with the highest AUC value as the pathological type diagnostic optimization model (PT_OM), and outputs and saves the original records.

[0026] c) The model training submodule trains and saves pathological disease models through PDS. After training is completed, the last 8 models are selected from the saved models for testing. That is, the model testing submodule is automatically started to test PDTDS, automatically selects the model with the highest AUC value as the pathological disease diagnosis optimization model (PD_OM), and outputs and saves the original records.

[0027] Preferably, in step four (2), the preset number of benign and malignant images is 125.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows: The integrated system of the present invention is designed using a model optimization method, which realizes the supervision of the model and can ensure the stability of the diagnostic performance of the integrated system; The integrated system of the present invention has continuous learning ability, and its diagnostic ability is continuously improved with the learning of each stage, ensuring the accuracy of the diagnostic results; The present invention adopts a model integration design, and the integrated system can output multiple diagnostic results, which plays a very important auxiliary role in helping physicians identify diseases. Attached Figure Description

[0029] Figure 1 This is a system block diagram of the present invention;

[0030] Figure 2 This is a system flowchart of the present invention;

[0031] Figure 3 This is a flowchart of the method of the present invention;

[0032] In the diagram: 1. Continuous Learning Subsystem; 11. Data Processing Module; 110. Image Classification Submodule; 111. Dataset Construction Submodule; 12. Optimized Model Generation Module; 120. Model Training Submodule; 121. Model Testing Submodule; 13. Diagnosis Module; 14. Data Storage Module; 140. Mass Benign / Malignant Database; 141. Pathological Type Database; 142. Pathological Disease Database; 143. Optimized Model Database; 2. Ultrasound PACS Subsystem. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Please see Figure 1This invention provides an embodiment of an optimized integrated system for real-time monitoring of a continuous deep learning model, comprising a continuous learning subsystem 1 and an ultrasound PACS subsystem 2. The continuous learning subsystem 1 includes a data processing module 11, an optimized model generation module 12, a diagnostic module 13, and a data storage module 14. The ultrasound PACS subsystem 2 is data-connected to the data processing module 11, which in turn is data-connected to the optimized model generation module 12, the diagnostic module 13, and the data storage module 14. Both the optimized model generation module 12 and the diagnostic module 13 are data-connected to the data storage module 14, and the diagnostic module 13 is data-connected to the ultrasound PACS subsystem 2. Block 11 includes an image classification submodule 110 and a dataset construction submodule 111; the preferred model generation module 12 includes a model training submodule 120 and a model testing submodule 121, and the model training submodule 120 and the model testing submodule 121 establish a data connection; the data storage module 14 includes a tumor benign / malignant database 140, a pathology type database 141, a pathological disease database 142 and a preferred model database 143, and the tumor benign / malignant database 140, the pathology type database 141 and the pathological disease database 142 all establish a data connection with the data processing module 11, and the preferred model database 143 establishes a data connection with the preferred model generation module 12 and the diagnosis module 13.

[0035] Please see Figure 2-3 Tables 1-3 illustrate an embodiment of the present invention: a method for implementing a preferred integrated system for real-time monitoring of a continuous deep learning model, comprising: step one, dataset construction; step two, algorithm optimization; step three, system development; and step four, system application.

[0036] In step one above, the dataset construction includes the following steps:

[0037] 1) Constructing the experimental dataset: 499 cases of breast ultrasound with pathological results were selected. The 499 data points were divided into 6 training datasets according to the order of the patients' examinations. These datasets were used to train the continuous deep learning model in stages. 965 images of benign breast masses and 965 images of malignant breast masses were collected. 800 images were randomly selected for training and 165 images were selected for testing. The experimental dataset (EDS) was constructed.

[0038] 2) Constructing a test dataset for benign and malignant tumor diagnosis: 130 cases were selected as the test dataset (TDS). Among the cases used for testing, 200 images of benign tumors and 200 images of malignant tumors were selected to construct a test dataset for benign and malignant tumor diagnosis (BMTDS).

[0039] 3) Construct a pathological type diagnostic test dataset: Based on the pathological type of the pathological diagnosis results, 200 images of invasive non-specific carcinoma were selected as the positive class, and 200 images of other pathological types were randomly selected as the negative class to construct a pathological type diagnostic test dataset (PTTDS).

[0040] 4) Construct a pathological disease diagnosis test dataset: Based on the pathological disease diagnosis results, 200 images of invasive ductal carcinoma of the breast were selected as the positive class, and 200 images of other pathological diseases were randomly selected as the negative class to construct a pathological disease diagnosis test dataset (PDTDS).

[0041] In step two above, the algorithm preferably includes the following steps:

[0042] 1) Selecting the best algorithms: Select 13 commonly used algorithms to develop diagnostic models. Train the models using the EDS obtained in step 1 above and test the models using BMTDS to obtain the AUC value (Area under the curve) of the diagnostic models. Compare the diagnostic performance of the models and select 5 algorithms with better diagnostic performance (see Table 1).

[0043] 2) Selecting the optimal algorithm: Five algorithms were compared with each other (see Table 2). Models were built using each of the five algorithms. BMTDS, PTTDS, and PDTDS were used to test and compare the performance of the algorithms in diagnosing benign and malignant masses, pathological types, and pathological diseases (see Table 3). Among the five algorithms, Inception_resnet_v2 had a total AUC of 2.161, which was higher than the other algorithms. Therefore, it was selected as the development algorithm for the system.

[0044] In step three above, based on the dataset constructed in step one and the optimal algorithm obtained in step two, a continuous learning subsystem 1 is developed using Python and integrated with the ultrasound PACS subsystem 2.

[0045] In step four above, the system application includes the following steps:

[0046] 1) Data input: The physician provides data and images to the data processing module 11 through the ultrasound PACS subsystem 2. When the image has pathological results, proceed to step 2); when the image does not have pathological results, proceed to step 3.

[0047] 2) Continuous Learning: When the data processing module 11 receives images with pathological results, it automatically classifies the images and constructs datasets, including a benign and malignant mass dataset (BMS), a pathological type dataset (PTS), and a pathological disease dataset (PDS). When the number of benign and malignant images reaches 125, the optimal model generation module 12 automatically starts to train and test the models. Three sets of models are trained using the BMS, PTS, and PDS constructed in step 2). After training, the last eight models in each set are saved for testing. The model with the highest AUC value is automatically selected as the optimal model in that set. After all three models are trained, the model testing submodule 121 uses the three optimal models to perform a comprehensive test on the TDS and outputs the results. The specific steps of model training and testing are as follows:

[0048] a) The model training submodule 120 trains and saves the benign and malignant diagnostic model through BMS. After training is completed, the last 8 models are selected from the saved models for testing. That is, the model testing submodule 121 is automatically started to test BMTDS, and the model with the highest AUC value is automatically selected as the benign and malignant diagnostic optimization model (BM_OM), and the original records are output and saved.

[0049] b) The model training submodule 120 trains and saves the pathological type model through PTS. After training is completed, the last 8 models are selected from the saved models for testing. That is, the model testing submodule 121 is automatically started to test PTTDS, and the model with the highest AUC value is automatically selected as the pathological type diagnostic optimization model (PT_OM), and the original records are output and saved.

[0050] c) The model training submodule 120 trains and saves the pathological disease model through PDS. After training is completed, the last 8 models are selected from the saved models for testing. That is, the model testing submodule 121 is automatically started to test PDTDS, and the model with the highest AUC value is automatically selected as the pathological disease diagnosis optimization model (PD_OM), and the original records are output and saved.

[0051] 3) Automatic diagnosis: When the data processing module 11 receives an image without pathological results, the data processing module 11 transmits the image to the diagnosis module 13. The diagnosis module 13 first uses BM_OM to perform benign or malignant diagnosis on the mass image. In addition to obtaining benign or malignant diagnosis results, it selects the image with the highest probability of malignancy, performs pathological type diagnosis through PT_OM, and performs pathological disease diagnosis through PD_OM. After completing the diagnosis, it returns the results to the ultrasound PACS subsystem 2.

[0052] The integrated system provided in the above embodiments was subjected to diagnostic performance testing and evaluation. The same evaluation criteria (see Table 4) were used to compare the diagnostic capabilities of the integrated system (see Table 5) and the diagnostic capabilities of physicians (see Table 6). Twenty-one physicians who performed breast ultrasound examinations during their work participated in the testing. The evaluation criteria used nine indicators: sensitivity, specificity, and AUC value for benign / malignant diagnosis; diagnostic accuracy of pathological type (DAPT); accuracy of pathological type identification (APTI); missed diagnosis rate of pathological type (MDRPT); diagnostic accuracy of pathological diseases (DAPD); accuracy of differentiating pathological diseases (ADPD); and missed diagnosis rate of pathological diseases. Diseases (MDRPD), the scoring standard is up to 100 points, the AUC value is 20 points, and other indicators are 10 points. Three pathological types can be selected, and three indicators are used for evaluation. APTI refers to the correct diagnosis among the three diagnoses. Different scores are given according to the ranking of the correct diagnosis: 3 points for the first place, 2 points for the second place, 1 point for the third place, and 0 points for not being among the three diagnoses. The cumulative score is the number of cases * 3, and APTI is the ratio of the cumulative score to the total score. ADPD is similar, with six diagnoses selectable. The correct diagnosis ranks first and receives 6 points, and so on, with 0 points for not being among the six diagnoses. The cumulative score is the number of cases * 6. The number of missed diagnoses is the number of cases that receive 0 points. The lower the MDRPD and MDRPT scores, the higher the score. The integrated system and physicians' diagnostic capabilities are ranked (see Table 7).

[0053] Table 1. Comparison of diagnostic performance of 13 algorithms

[0054]

[0055]

[0056] Table 2. Comparison of diagnostic performance among the five algorithms

[0057]

[0058] Table 3. Comprehensive comparison results of the five algorithms

[0059]

[0060] Table 4. Case Test Evaluation Criteria

[0061]

[0062]

[0063] Table 5 Evaluation results of the integrated system after 6 training tests.

[0064]

[0065] Table 6. Evaluation results of 21 physicians

[0066]

[0067]

[0068] Table 7 Ranking of Integrated Systems and Physician Diagnostic Capabilities

[0069]

[0070]

[0071] Based on the above, the advantages of this invention are as follows: When in use, the ultrasound PACS subsystem 2 first provides data and images to the continuous learning subsystem 1, which is then received by the data processing module 11. When the received image contains pathological results, the image classification submodule 110 automatically classifies the image, and the dataset construction submodule 111 automatically constructs the dataset and stores it in the tumor benign / malignant database 140, the pathological type database 141, and the pathological disease database 142, respectively. When the number of benign and malignant images reaches a set value, the optimal model generation module 12 automatically starts, the model training submodule 120 performs model training, the model testing submodule 121 performs model testing, and the three optimal models obtained are stored in the optimal model database 143. When the data processing module 11 receives an image without pathological results, it transmits the image to the diagnosis module 13. The diagnosis module 13 calls the optimal model in the data storage module 14 for diagnosis and returns the result to the ultrasound PACS subsystem 2. After model training, the system automatically optimizes the model selection process and tests it with external data. The model with the highest AUC is selected as the OM (Model-Oriented Model), thus achieving model supervision and ensuring stable diagnostic performance. It also possesses continuous learning capabilities, allowing the model's diagnostic ability to continuously improve. Through model integration, the integrated system can output multiple results, including whether the mass is benign or malignant, three pathological type diagnoses (ranked by probability), and six pathological disease diagnoses (also ranked by probability). These results are highly valuable for physicians' diagnoses. To ensure that the results do not interfere with each other, benign / malignant diagnoses, pathological types, and pathological disease diagnoses are independent of each other, preventing errors in subsequent diagnoses due to errors in previous ones. This design aligns with physicians' diagnostic thinking. The multiple outputs of pathological types and pathological disease diagnoses also align with physicians' diagnostic thinking. When diagnosing a disease, physicians need to consider other differential diagnoses in addition to the primary diagnosis. When there are too many diseases to consider, the integrated system outputs the most likely multiple diagnostic results for physicians' reference, helping them to differentiate diseases.

[0072] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for implementing a preferred integrated system for real-time monitoring of a continuous deep learning model, the system comprising a continuous learning subsystem (1) and an ultrasound PACS subsystem (2), characterized in that: The continuous learning subsystem (1) includes a data processing module (11), a preferred model generation module (12), a diagnostic module (13), and a data storage module (14). The ultrasound PACS subsystem (2) is connected to the data processing module (11). The data processing module (11) is connected to the preferred model generation module (12), the diagnostic module (13), and the data storage module (14). The preferred model generation module (12) and the diagnostic module (13) are both connected to the data storage module (14). The diagnostic module (13) is connected to the ultrasound PACS subsystem (2). The acoustic PACS subsystem (2) establishes a data connection. The data storage module (14) includes a tumor benign and malignant database (140), a pathological type database (141), a pathological disease database (142), and a preferred model database (143). The tumor benign and malignant database (140), the pathological type database (141), and the pathological disease database (142) are all connected to the data processing module (11). The preferred model database (143) is connected to the preferred model generation module (12) and the diagnosis module (13). The implementation method includes: Step 1, dataset construction; Step 2, algorithm optimization; Step 3, system development; and Step 4, system application. In step one above, the dataset construction includes the following steps: 1) Constructing the experimental dataset: 499 cases of breast ultrasound with pathological results were selected for training. 965 images of benign breast masses and 965 images of malignant breast masses were collected. 800 images were randomly selected for training and 165 images were selected for testing to construct the experimental dataset (EDS). 2) Constructing a test dataset for benign and malignant tumor diagnosis: 130 cases were selected as the test dataset (TDS). Among the cases used for testing, 200 images of benign tumors and 200 images of malignant tumors were selected to construct a test dataset for benign and malignant tumor diagnosis (BMTDS). 3) Construct a pathology type diagnostic test dataset: Based on the pathology type of the pathology diagnosis results, 200 images of invasive non-specific carcinoma were selected as the positive class, and 200 images of other pathological types were randomly selected as the negative class to construct a pathological type diagnostic test dataset (PTTDS). 4) Construct a pathological disease diagnosis test dataset: Based on the pathological disease diagnosis results, 200 images of invasive ductal carcinoma of the breast were selected as the positive class, and 200 images of other pathological diseases were randomly selected as the negative class to construct a pathological disease diagnosis test dataset (PDTDS). In step two above, the algorithm preferably includes the following steps: 1) Select the best algorithm: Select 13 commonly used algorithms to develop diagnostic models. Train the model using the EDS obtained in step 1 above and test the model using BMTDS to obtain the AUC value (Area under the curve) of the diagnostic model. Compare the diagnostic performance of the models and select 5 algorithms with better diagnostic performance. 2) Selecting the optimal algorithm: Five algorithms were compared with each other, and models were built using each of the five algorithms. BMTDS, PTTDS, and PDTDS were used to test and compare the performance of the algorithms in diagnosing benign and malignant masses, pathological types, and pathological diseases. Among these five algorithms, the algorithm with the highest AUC value was selected as the development algorithm for the system. In step three above, based on the dataset constructed in step one above and the optimal algorithm obtained in step two, a continuous learning subsystem (1) is developed using Python and integrated with the ultrasound PACS subsystem (2). In step four above, the system application includes the following steps: 1) Data input: The physician provides data and images to the data processing module (11) through the ultrasound PACS subsystem (2). When the image is an image with pathological results, proceed to step 2); when the image is an image without pathological results, proceed to step 3. 2) Continuous learning: When the data processing module (11) receives an image with pathological results, it automatically classifies the image and constructs a dataset, respectively constructing a benign and malignant mass dataset (BMS), a pathological type dataset (PTS), and a pathological disease dataset (PDS); when the number of benign and malignant images reaches a preset number, the optimal model generation module (12) automatically starts to train and test the model. It trains three sets of models using the BMS, PTS, and PDS constructed in step 2). After each set of models is trained, the last eight models in the set are saved for testing. The model with the highest AUC value is automatically selected as the optimal model in the set. When all three models are trained, the model testing submodule (121) uses the three optimal models to perform a comprehensive test on the TDS and outputs the results; the specific steps of model training and testing are as follows: a) The model training submodule (120) trains and saves the benign and malignant diagnostic model through BMS. After training is completed, the last 8 models are selected from the saved models for testing. That is, the model testing submodule (121) is automatically started to test BMTDS. The model with the highest AUC value is automatically selected as the benign and malignant diagnostic optimization model (BM_OM), and the original records are output and saved. b) The model training submodule (120) trains and saves the pathological type model through PTS. After training is completed, the last 8 models are selected from the saved models for testing. That is, the model testing submodule (121) is automatically started to test PTTDS, and the model with the highest AUC value is automatically selected as the pathological type diagnostic optimization model (PT_OM), and the original records are output and saved. c) The model training submodule (120) trains and saves the pathological disease model through PDS. After training is completed, the last 8 models are selected from the saved models for testing. That is, the model testing submodule (121) is automatically started to test PDTDS, and the model with the highest AUC value is automatically selected as the pathological disease diagnosis optimization model (PD_OM), and the original records are output and saved. 3) Automatic diagnosis: When the data processing module (11) receives an image without pathological results, the data processing module (11) transmits the image to the diagnosis module (13). The diagnosis module (13) first uses BM_OM to perform benign or malignant diagnosis on the mass image. In addition to obtaining benign or malignant diagnosis results, it selects the image with the highest probability of malignancy, performs pathological type diagnosis through PT_OM, performs pathological disease diagnosis through PD_OM, and returns the results to the ultrasound PACS subsystem (2) after completing the diagnosis.

2. The implementation method of the preferred integrated system for real-time monitoring of continuous deep learning models according to claim 1, characterized in that: The data processing module (11) includes an image classification submodule (110) and a dataset construction submodule (111).

3. The implementation method of the preferred integrated system for real-time monitoring of continuous deep learning models according to claim 1, characterized in that: The preferred model generation module (12) includes a model training submodule (120) and a model testing submodule (121), and the model training submodule (120) and the model testing submodule (121) establish a data connection.

4. The implementation method of the preferred integrated system for real-time monitoring of continuous deep learning models according to claim 1, characterized in that: In step 1), the 499 data points are divided into 6 training datasets according to the order of the patient's examination, which are used to train the continuous deep learning model in stages.

5. The implementation method of the preferred integrated system for real-time monitoring of continuous deep learning models according to claim 1, characterized in that: In step four (2), the preset number of benign and malignant images is 125.

Images