Federated Learning of Medical Verification Models

Federated learning of medical validation models addresses inefficiencies and accuracy issues by training across multiple sites, enhancing validation model performance and security.

JP2024542035AActive Publication Date: 2024-11-13F HOFFMANN LA ROCHE & CO AG
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2024525719
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2021-11-01
Publication Date
2024-11-13
Estimated Expiration
2041-11-01

AI Technical Summary

Technical Problem

Current medical validation models are resource-consuming and lack efficiency due to local training, and may not generalize accurately across different medical sites, leading to potential misdiagnosis risks.

Method used

Implement federated learning across multiple computing nodes to jointly train a medical validation model using local datasets while maintaining data privacy and security, where a master node provides definition information for training and aggregates results to determine a final model.

Benefits of technology

Enhances the accuracy and quality of medical data validation by leveraging diverse local datasets while ensuring data security and privacy, reducing manual effort and improving model generalization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024542035000001_ABST
    Figure 2024542035000001_ABST
Patent Text Reader

Abstract

A computer-implemented method is provided that includes sending, by a master node, definition information regarding an initial medical validation model to a plurality of computing nodes (410), performing, by the master node, a federated learning process with the plurality of computing nodes (420) to jointly train the initial medical validation model using respective processed local training datasets available at the plurality of computing nodes, where the respective local training datasets are processed by the plurality of computing nodes based on the definition information, and determining, by the master node, a final medical validation model based on a result of the federated learning process (430). Through this solution, federated learning addresses data security and privacy concerns due to local site ownership.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE Embodiments of the present disclosure relate generally to the field of computer science, and in particular to methods, devices, and computer program products for federated learning of medical validation models. [Background technology]

[0002] Medical tests are performed almost daily in medical laboratories, from which numerous medical test reports are generated and present medical data. Before releasing the medical test report to the department or the patient, a validation procedure is initiated to ensure that the medical data presented in the report is valid in order to avoid misdiagnosis of the patient. However, despite the introduction of some automated features, the current validation procedure requires a lot of effort.

[0003] With the development of machine learning, currently, it is proposed to train machine learning models for automatic validation of medical data. Training of machine learning models may require training data including past medical data. Traditionally, each laboratory or hospital may collect locally generated past medical data to train the machine learning models to be used. However, such individual local training of models may consume resources and may not be very efficient. Meanwhile, the available past medical data may be limited at each local site. For example, most of the medical data collected at a health checkup center may reflect the disease conditions of healthy subjects, while most of the medical data collected at an oncology clinic may reflect the disease conditions of tumor patients. Thus, a medical validation model trained at one local site may not be generalized to provide accurate validation results for other local sites.

[0004] Therefore, it is desirable to provide a solution for efficiently and effectively training medical validation models. Summary of the Invention

[0005] Generally, exemplary embodiments of the present disclosure provide a solution for federated learning of medical validation models.

[0006] In a first aspect, a computer-implemented method is provided, the method including: sending, by a master node, definition information for an initial medical validation model to a plurality of computing nodes; performing, by the master node, a federated learning process with the plurality of computing nodes to jointly train the initial medical validation model using respective processed local training datasets available at the plurality of computing nodes, where the respective local training datasets are processed by the plurality of computing nodes based on the definition information; and determining, by the master node, a final medical validation model based on a result of the federated learning process.

[0007] In a second aspect, a computer-implemented method is provided that includes receiving, by a computing node, definition information from a master node regarding an early medical validation model, processing a local training dataset based at least on the definition information, and performing a federated learning process with the master node and at least one further computing node to jointly train an early medical validation model using the processed local training dataset.

[0008] In a third aspect, there is provided an electronic device comprising at least one processor and at least one memory comprising computer readable instructions that, when executed by the at least one processor of the electronic device, cause the electronic device to perform the steps of the method in the first aspect described above.

[0009] In a fourth aspect, there is provided an electronic device comprising at least one processor and at least one memory comprising computer readable instructions that, when executed by the at least one processor of the electronic device, cause the electronic device to perform the steps of the method in the second aspect described above.

[0010] In a fifth aspect, there is provided a computer program product comprising instructions which, when executed by a processor of an apparatus, cause the apparatus to perform any one of the steps of the method of the first aspect described above.

[0011] In a sixth aspect, there is provided a computer program product comprising instructions which, when executed by a processor of an apparatus, cause the apparatus to perform any one of the steps of the method of the second aspect described above.

[0012] It should be understood that the Summary section is not intended to identify key or essential features of the embodiments of the present disclosure, nor is it intended to be used to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood by the following description. [Brief description of the drawings]

[0013] The following detailed description of the embodiments of the present disclosure can be best understood when read in conjunction with the following drawings.

[0014] [Figure 1] FIG. 1 illustrates an exemplary environment in which embodiments of the present disclosure may be implemented. [Diagram 2] FIG. 1 is a block diagram of a system for federated learning and application of medical validation models, according to some embodiments of the present disclosure. [Diagram 3] FIG. 3 is a block diagram of the computing nodes and the master node in the system of FIG. 2 for federated learning of medical validation models in accordance with some embodiments of the present disclosure. [Figure 4]1 is a flowchart of an example process for training a medical validation model implemented at a master node, according to some embodiments of the present disclosure. [Diagram 5] 1 is a flowchart of an example process for training a medical validation model implemented on a computing node according to some embodiments of the present disclosure. [Figure 6] FIG. 1 is a block diagram of an example computing system / device suitable for implementing example embodiments of the present disclosure.

[0015] Throughout the drawings, the same or similar reference numbers represent the same or similar elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Next, the principles of the present disclosure will be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only, and are intended to help those skilled in the art to understand and practice the present disclosure, without implying any limitation on the scope of the present disclosure. The disclosure described herein can be implemented in various ways other than those described below.

[0017] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0018] References in this disclosure to "one embodiment," "embodiment," "exemplary embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but not all embodiments need to include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Moreover, if a particular feature, structure, or characteristic is described in connection with an exemplary embodiment, it is believed to be within the knowledge of one of ordinary skill in the art to affect such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described.

[0019] In this specification, terms such as "first" and "second" may be used to describe various elements, but it should be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0020] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the exemplary embodiments. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that the terms "comprises", "comprising", "has", "having", "includes" and / or "including", as used herein, specify the presence of stated features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0021] As used herein, the term "model" refers to an association between an input and an output learned from training data, and thus may generate a corresponding output for a given input after training. The generation of the model may be based on machine learning techniques. Machine learning techniques may also be referred to as artificial intelligence (AI) techniques. In general, machine learning models may be constructed that receive input information and make predictions based on the input information. For example, a classification model may predict the class of the input information from among a set of predefined classes. As used herein, a "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and are used interchangeably herein.

[0022] In general, machine learning may typically include three stages: a training stage, a validation stage, and an application stage (also called an inference stage). In the training stage, a given machine learning model may be iteratively trained using a large amount of training data until the model can obtain consistent inferences from the training data similar to those that human intelligence can make. Through training, the machine learning model may be considered to be able to learn associations between inputs and outputs (also called input-output mappings) from the training data. A set of parameter values ​​for the trained model is determined. In the validation stage, a validation input is applied to the trained machine learning model to test whether the model can provide a correct output to determine the performance of the model. In the application stage, the resulting machine learning model may be used to process actual model inputs based on the set of parameter values ​​obtained from training to determine the corresponding model output.

[0023] Example Environment As mentioned above, validation procedures are important to ensure the validity of medical data generated in various medical tests. FIG. 1 illustrates an environment 100 in which an exemplary embodiment of the present disclosure can be implemented. The environment 100 includes a typical workflow for medical diagnostic tests performed at different local sites 105-1, 105-2, ... 105-N, where N is 1 or more. Hereinafter, for the sake of explanation, the local sites 105-1, 105-2, ... 105-N are collectively or individually referred to as local sites 105. Hereinafter, the 105-N are collectively or individually referred to as local sites 105. The local sites 105 may include medical laboratories, hospitals, medical departments, physical examination centers, medical institutions, or any site where medical tests are performed and medical data obtained from the medical tests needs to be validated.

[0024] 1, for the sake of simplicity, the medical diagnostic test workflow is detailed at local site 105-1, but it will be understood that similar workflows are performed at other local sites. The workflow generally includes performing a medical test on a test sample for medical diagnosis, generating medical data in the medical test, and validating the generated medical data.

[0025] As shown in FIG. 1, at the local site 105, a medical testing system 110 is configured to perform medical tests on a test sample 102 and generate medical data 112 associated with the test sample 102. The medical tests may include in vitro diagnostic tests, such as biochemical detection tests or immunodetection tests. The medical testing system 110 may include one or more automated testing instruments or analyzers designed for analysis of the test sample by various chemical, biological, physical, or other medical testing procedures. In some examples, the instruments or analyzers may be configured to induce a reaction between the sample and a reagent to obtain a measurement value. Examples of such instruments or analyzers include clinical chemistry analyzers, coagulation analyzers, immunochemistry analyzers, hematology analyzers, urine analyzers, and nucleic acid analyzers, which are used for qualitative and / or quantitative detection of analytes present in a sample, detect the results of a chemical or biological reaction, and / or monitor the progress of a chemical or biological reaction.

[0026] The medical testing system 110 may be operable to perform a medical test to measure a parameter of a sample or at least one analyte thereof. A medical test may include one or more tests performed on the sample 102. The medical testing system 110 may return test results corresponding to each test as medical data 112. Test results that may be returned by the medical testing system 110 may be obtained by determining the concentration of the analyte in the sample, a digital (yes or no) result indicating the presence of the analyte in the sample (corresponding to a concentration above a detection level), data obtained from mass spectrometry of proteins or metabolites, and / or various types of physical, mechanical, optical, electrical or chemical parameters, etc.

[0027] Some specific examples of types of test items may include levels of alanine aminotransferase (ALT), aspartate aminotransferase (AST), glutamate dehydrogenase (GLDH), sodium (NA) concentration, age, hemoglobin, plasma proteins, albumin (ALB), globulin (GLB), total bilirubin (TBIL), direct bilirubin (DBIL), total bile acids (TBA), blood urea nitrogen (BUN), etc. The examples listed here are not exhaustive. The test items to be performed in a particular medical test may be specified by the entity requesting the medical test, such as a medical department, a medical examination center, a physician, or a patient.

[0028] The test sample 102, sometimes referred to as a biological sample, is a biological material suspected of containing one or more analytes of interest, the qualitative and / or quantitative detection of which may be correlated to a clinical condition. Biological samples are derived from biological sources such as physiological fluids including blood, saliva, ocular lens fluid, cerebrospinal fluid, sweat, urine, stool, semen, milk, peritoneal fluid, mucus, synovial fluid, peritoneal fluid, amniotic fluid, tissues, cells, etc. Such biological sources may be collected from biological subjects, e.g., patients, humans, animals, etc.

[0029] Biological samples may be pre-treated before use, such as preparing plasma or serum from blood. Processing methods may include centrifugation, filtration, distillation, dilution, concentration and / or separation of sample components, including the analyte of interest, inactivation of interfering components, and addition of reagents. Biological samples may be used directly as derived from the source, or may be used after pre-treatment to modify the characteristics of the sample. In some embodiments, solid or semi-solid biological materials may first be made liquid by dissolving or suspending in a suitable liquid medium.

[0030] The term "reagent" refers to a substance that is added to a biological sample when performing a particular medical test on the biological sample to induce a particular reaction in the sample. A reagent may be specific to a particular test or assay. For example, in the context of measuring the partial clot formation time of a blood sample, the analyzer may be configured to add an activator as a reagent to the blood sample to activate the intrinsic coagulation pathway. A particular substance may be a "modifier" or a "reagent" in different contexts. In some instances, a reagent may not be added to the biological sample being tested.

[0031] The medical data 112 associated with the test sample 102 may include one or more test results of a test item performed in a medical test at the medical testing system 110. The type of test result may be specified by an operator (e.g., a laboratory technician) of the medical testing system 110 or may be automatically identified from an electronic order via an information system connected to the medical testing system 110. In some examples, the medical data 112 may be organized into a medical test report that lists specific test items and corresponding test results. In some examples, in addition to the test results generated in the medical test, the medical data 112 may also include auxiliary information, such as information related to the test sample 102 and / or the biological subject (e.g., a patient) from which the test sample 102 is collected.

[0032] The medical data 112 is provided to a verification system 120 to assess the legitimacy of the medical data 112 and determine whether the medical data 112 can be released. Verification is necessary because many potential problems can occur during the sample collection and testing process. For example, a patient sample may be mislabeled, resulting in test results being reported in relation to the wrong patient. As another example, a patient sample may be improperly drawn or improperly handled, resulting in sample contamination and erroneous test results. Additionally, a laboratory analyzer may malfunction or drift out of calibration, again causing the analyzer to report erroneous results.

[0033] To improve the efficiency of the medical validation process, trained medical validation models 130 may be utilized in the validation system 120 to automatically evaluate the validity of the medical data 112. The medical validation models 130 are trained to automatically process input medical data and output a validation result indicative of one of the validation categories. Each trained medical validation model 130 represents an association between medical data and a validation category.

[0034] The input to the medical validation model 130 is medical data, and the output validation result 122 from the medical validation model 130 includes one of the validation categories. In some embodiments, the medical validation model 130 may be designed as a classification model to classify / assign the input medical data into one of the validation categories. In some embodiments, the validation result 122 from the medical validation model 130 may include an explicit indication of the validation category and / or the confidence level of the validation category for the current medical data. In determining the output validation category, the medical validation model 130 measures the probability of each of the given validation categories and selects the one with the highest probability.

[0035] The medical data may include one or more test results for a test item, which may include a measurement for the test item and / or a digital (yes or no) result indicating the presence of a particular analyte in the test sample. The medical data 112 may further include other information, such as patient information, department information, etc.

[0036] Each of the validation categories output by the medical validation model may indicate one of the predefined actions to be performed on the medical data, which can be thought of as a suggestion for the system or user to automatically or manually determine how the medical data can be handled in the next step of the overall medical diagnostic test workflow.

[0037] Medical validation is finding potential errors in the medical data before it is released to an entity (such as a medical department or a patient) requesting a medical test. If the medical data is validated as correct and free of errors, the next step is to release the medical data to that entity (or request a quick manual review and then release to the entity). In this case, one possible action to be performed on the medical data is to release the medical data, either directly or after a quick manual review, to an entity requesting a medical test related to the medical data. The validation result 122 may include a validation category indicating that the medical data 112 is correct to be released directly (or released after a quick manual review) to a requester ordering a medical test.

[0038] In other cases, the medical data is verified as having errors due to the test sample, the medical diagnostic test procedure performed, the reagents used in the medical test, the inconsistency of the test sample with the physical state of the biological subject, insufficient information for decision making, etc. In such cases, it is necessary to take corresponding actions to correct the errors. The action shown in the medical data verification result 122 is to suggest further verification of the medical data. This action is a general suggestion, which means that the current medical data should not be made public and a manual review is required to determine how the medical data can be further verified.

[0039] In some examples, one or more specific actions for further validation may be indicated by a validation category output from the medical validation model 130, including an action to rerun a medical test associated with the medical data, an action to verify the patient's medical history, an action to verify the reaction of a reagent in the medical test, such as verifying a reagent response curve, an action to verify a test sample collected for use in the medical test, an action to verify the medical data in conjunction with a clinical diagnosis, an action to verify medication usage of the patient, etc. It will be appreciated that the above next step actions are merely some specific examples, and more, less, or different actions may be specified as needed in the actual use case and thus in the validation category of the medical validation model 130.

[0040] Principle of Operation and Exemplary Systems The introduction of medical validation models can significantly reduce the manual effort spent on reviewing medical data, and can also improve the accuracy and quality of medical data validation. However, for various reasons, currently, each local site (e.g., laboratory or hospital) trains its own medical validation model using locally collected medical data, which may be resource-consuming and not very efficient. Meanwhile, medical validation models trained at a local site may not generalize to provide accurate validation results for other local sites.

[0041] A simple solution is to collect historical medical data from different local sites and train a model in a central node. However, it is not practical and ideal considering the confidentiality of medical data and poor networking connections between different local sites. Some local sites may refuse to export medical data due to some agreements or regulations.

[0042] According to an exemplary embodiment of the present disclosure, a solution for federated learning of medical validation is proposed. In this solution, a master node and multiple computing nodes cooperate to execute a federated learning process to jointly train a medical validation model. The master node provides definition information about the medical validation model to the computing nodes. The computing nodes process local training datasets respectively based on the definition information, and utilize the processed local training datasets in the federated learning process. Therefore, the local training datasets in the computing nodes themselves do not need to be disclosed to the master node or other computing nodes. The master node determines a final medical validation model based on the result of the federated learning process.

[0043] Through this solution, federated learning addresses data security and privacy concerns due to local sites owning the training dataset for model training. Furthermore, the final medical validation model is trained on a different local training dataset, which allows for improving the accuracy and quality of medical validation using the model.

[0044] Exemplary embodiments of the present disclosure will now be described with reference to the drawings. First, reference is made to Fig. 2, which illustrates a system 200 for federated learning and application of medical validation models. The system 200 of Fig. 2 may be partially implemented in the environment 100 of Fig. 1. For purposes of explanation, reference is made to Fig. 1 to describe the system 200.

[0045] As shown, system 200 includes a master node 202 and a number of computing nodes 210-1, 210-2, ..., 210-N (hereinafter collectively or individually referred to as computing nodes 210). Master node 202 and computing nodes 210 may comprise or be implemented as any number of devices / systems having computing capabilities, such as servers, computers, mainframes, etc.

[0046] Computing nodes 210-1, 210-2, ..., 210-N may be located at local sites 105-1, 105-2, ..., 105-N, respectively, and may access data otherwise available at the local sites. Thus, computing nodes 210-1, 210-2, ..., 210-N may be considered local nodes with respect to those sites. As shown in FIG. 2, computing node 210-1 may access data stored in database 220-1 available at local site 105-1, computing node 210-2 may access data stored in database 220-2 available at local site 105-2, computing node 210-N may access data stored in database 220-N available at local site 105-N, and so on. For purposes of explanation, databases 220-1, 220-2, ..., 220-N may be collectively or individually referred to as databases 220.

[0047] As described in detail below, according to an embodiment of the present disclosure, the master node 202 and the multiple computing nodes 210 cooperate to jointly train the early medical validation model 230 by federated learning. The computing nodes 210 obtain their respective local training datasets for federated learning from their accessible databases 220. In such a case, the local sites 105 may be referred to as contributing sites because they provide data to the training of the global model. The computing nodes 210 process the local training datasets based on the definition of the early medical validation model 230 received from the master node 202. The definition information may define one or more aspects of the input and output of the early medical validation model 230 to be trained. Using the definition information, the local training dataset may be adapted at the computing node to be suitable for training the global model. The master node 202 may determine the final medical validation model 240 based on the results of the federated learning.

[0048] Federated learning is a machine learning technique that trains a model across multiple distributed nodes that hold a training dataset. Federated learning allows the computing nodes 210 to collaboratively learn a model while keeping all the training dataset on the node. That is, the local training dataset is not exposed to other computing nodes 210 or the master node 202 during training. As a result, the local site 105 is free of privacy concerns and data ownership concerns since the raw medical data never leaves the local computing node 210. Additionally, security concerns are greatly reduced since there is no single node where a security breach could put large amounts of raw data at risk.

[0049] The master node 202 and the computing nodes 210 may be deployed with respective federated learning engines to perform the federated learning process for the early medical validation model 230. There are various federated learning frameworks that may be applied to the embodiments of the present disclosure. As some examples, applicable federated learning frameworks may include Tensorflow Federated (TFF), Pysyft, or Federated AI Technology Enabler (FATE), and any other federated learning frameworks currently available or developed in the future.

[0050] To support federated learning among the nodes, in the system 200, the master node 202 is communicatively connected to a plurality of computing nodes 210. In some embodiments, a star topology network may be established between the master node 202 and the computing nodes 210. In some embodiments, the star topology network allows outgoing connections from each computing node 210 to the master node 202, but does not allow incoming requests to each computing node 210. The outgoing connections can further ensure data security at the computing nodes 210.

[0051] In some embodiments, the resulting final medical validation model 240 may be distributed to a local site for use in medical validation. A local site that receives the final medical validation model 240 for use may be referred to as a consumer site. In some embodiments, the final medical validation model 240 may be distributed to one or more sites other than the local site 255 that serves as a contributing site, such as the local site 105 as shown in FIG. 2. For example, the master node 202 may distribute the final medical validation model 240 to the computing node 250 of the local site 255. In some embodiments, the master node 202 may alternatively or additionally distribute the final medical validation model 240 to one or more of the local sites 105 that contribute training data for model training.

[0052] Federated Learning of Medical Verification Models 3 illustrates a block diagram of the computing nodes 210 and the master node 202 in the system 200 of FIG. 2 for federated learning of medical validation models, according to some embodiments of the present disclosure. For the sake of brevity, an exemplary detailed structure of one computing node 210 and the interaction between the master node 202 and this computing node 210 are illustrated in FIG. 3. It should be noted that each of the computing nodes 210 involved in the federated learning may include the same or similar components as illustrated in the computing node 210 of FIG. 3.

[0053] As shown, the master node 202 comprises a model configuration module 310 for configuring an early medical validation model 230 to be trained on a plurality of computing nodes, and a training aggregation module 330 for performing a federated learning process with the plurality of computing nodes 210 and aggregating intermediate training results during the federated learning process. The computing nodes 210 comprise a data preparation module 320 for pre-processing data from the database 220 that at least partially forms a training dataset for model training, and a local model training module 340 for performing the federated learning process based on the training dataset prepared by the data preparation module 320.

[0054] In some embodiments, the master node 202 and the multiple computing nodes 210 may perform a validation phase for machine learning to evaluate the performance of the trained medical validation model 305 determined from the federated learning process to determine a final medical validation model 240 for distribution. The master node 220 may include a model validation module 350, and the computing nodes 210 may include a local model validation module 360 ​​for performing the validation phase of the trained medical validation model 305.

[0055] The modules of the master node 202 and the computing nodes 210 may be implemented as one or more software engines, hardware components, middleware components, etc., comprised of logic for implementing the functionality attributed to a particular module.

[0056] A detailed description of each module of the master node 202 and the computing nodes 210 is provided below.

[0057] The model configuration module 310 of the master node 202 is configured to send definition information 312 regarding the initial medical validation model 230 to a computing node 210, such as to a data preparation module 320 of the computing node 210. The definition information is used to define the initial medical validation model 230 globally among the computing nodes 210. The initial medical validation model 230 may be defined similarly to the medical validation model 130 described with reference to Figure 1. As used herein, an "initial" medical validation model indicates that the medical validation model has initial parameter values ​​that may be iteratively updated during a training process.

[0058] In some embodiments, the definition information 312 may define one or more aspects of the inputs and outputs of the early medical validation model 230 to be trained. In some embodiments, the definition information 312 may further define the model configuration of the early medical validation model 230, including the model type, layers, processing units within the layers, and connections between processing units within the early medical validation model 230.

[0059] The data preparation module 320 of each computing node 210 is configured to obtain a local training dataset 302 from the database 220, process the local training dataset 302 based on the definition information 312 to obtain a processed local training dataset 322, and provide it to the local model training module 340.

[0060] Because the early medical validation model 230 is defined by the master node 202 globally across different local sites 105, the local training dataset available at the local sites 105 may not be suitable for training the early medical validation model 230. In addition to defining the model configuration, the definition information 312 from the master node 202 may at least enable the computing nodes 210 to prepare the local training dataset ready to train the early medical validation model 230.

[0061] As described above, the input to the initial medical validation model 230 may include medical data, and the output (i.e., the validation result) from the initial medical validation model 230 may indicate one of a number of validation categories corresponding to a predetermined action to be performed on the input medical data.

[0062] In some embodiments, the medical validation model 130 may be trained locally in a supervised manner. Thus, the local training dataset 302 at the local site 105 may include past medical data generated in medical exams and associated labeling information. The past medical data may include several medical exam reports generated in different medical exams of one or more patients.

[0063] The medical test report may indicate the item name indicating the item name and corresponding item value, including the test item name and the corresponding test value, auxiliary information such as information about the test sample 102 and / or the biological object (such as a patient) from which the test sample 102 was collected. The labeling information indicates the respective local validation category corresponding to the past medical data. The labeling information may be used as a ground truth validation category in training. In general, the labeled local validation category of each local site 105 may indicate an action that was considered to be appropriate for the past medical data and / or an action manually marked by a laboratory expert.

[0064] In some cases, in the local training dataset 302, different local sites 105 may use different item names to identify the same item in the historical medical data. For example, a local site 105 may record a test item with the item name "serum total prostate specific antigen", and other local sites 105 may record the same test item with the item code "tPSA" or "PSA". To avoid the medical validation model treating the same item with different item names as different items, in some embodiments, the model configuration module 310 of the master node 202 may determine the definition information 312 to indicate a unified item name in the medical data input to the initial medical validation model 230. According to the definition information 312, the data preparation module 320 of the commuting node 210 may map the local item names used in the historical medical data of the local training dataset 302 to the unified item names. That is, the data preparation module 320 may identify local item names that identify the same items as the unified item names indicated by the definition information 312, and if the local item names differ from the unified item names, replace the local item names in the historical medical data with the corresponding unified item names.

[0065] Table 1 shows an example of a mapping between uniform item names and local item names.

[0066] [Table 1]

[0067] In the example of Table 1, the local item names in the historical medical data available at local sites 105-1 and 105-2 have the same local item names as the unified test item names. For local site 105-N, the local item names "TestCode1", "TestCode2", "TestCode3", and "TestCode4" refer to the same items as the unified item names "TestItem4", "TestItem5", "TestItem...", and "TestItemn" shown in the definition information, respectively. The computing node 210 of local site 105-N may update the local training dataset 302 by replacing the local item names with the unified item names.

[0068] In some embodiments, the definition information 312 may indicate uniform item names for all possible items included in the input to the initial medical validation model 230. If the historical medical data in the local training dataset 302 available at the local site 105 does not include such local items or local item names, the data preparation module 320 may also be capable of including missing items with uniform item names in the input to the initial medical validation model 230.

[0069] In some embodiments, if an item is indicated by the definition information 312 at the local site 105 as forming an input to the early medical validation model 230, but a value for the indicated item is not available from historical medical data in the local training dataset 302, the corresponding computing node 210 may need to process the historical medical data to make it suitable for the early medical validation model 230. Because the early medical validation model 230 is trained globally, the input may generally include a large number of items considered to be relevant to the validation category.

[0070] The medical data acquired at the local sites 105 may not include all items, which may result in a sparse matrix issue and lower accuracy of the resulting model. For example, for the same medical test, some local sites 105 may record values ​​for five test items, while other local sites 105 may record values ​​for ten test items. The input to the initial medical validation model 230 may indicate more test items than some of the local sites. Also, from Table 1, it can be seen that at local site 105-1, test items "TestItem4", "TestItem5", "TestItem..." are missing, whereas at local site 105-2, test items "TestItem4", "TestItemn" are missing.

[0071] In some embodiments, to address the sparse matrix problem, a computing node 210 (i.e., a data preparation module 320 therein) may process its local training dataset 302 by filling in pre-defined values ​​for items that are not available in the local historical medical data but are required to be included as input to the initial medical validation model 230.

[0072] The predetermined value for an item may be determined in various ways. In some embodiments, the predetermined value may be determined as the average value of the reference value range of the indicated item. The reference value range is used to identify the normal situation of the indicated item, and any value lower than the lower limit or higher than the upper limit of the reference value range may be considered an outlier. The use of the average value of the reference value range may not affect the validation results of the medical data in which the indicated item is included. In some embodiments, the predetermined value may be determined as the median value of the available values ​​of the indicated item in the past medical data generated in other medical tests. For example, among all the past medical data generated from multiple medical tests, the value of the indicated item may be missing from one or some of the medical tests. In such a case, the other available values ​​of the same item may be used to determine the predetermined value to be filled in.

[0073] In some embodiments, the predefined value for an item may be determined in many other ways, such as a fixed value configured by the master node 202. In some embodiments, instead of filling in predefined values ​​for missing items, the computing node 210 may process the past medical data by marking the missing items as unchecked. In some embodiments, the computing node 210 (i.e., the data preparation module 320) may address the sparse matrix issue by marking the values ​​for the missing items as extreme values ​​(e.g., −9999) and converting the sparse matrix constructed from the input past medical data into a dense matrix. The computing node 210 may apply any suitable technique to convert the sparse matrix into a dense matrix, one example of which may include principal component analysis.

[0074] In some embodiments, some items in the medical data input to the initial medical validation model 230 may have numerical values. Different local sites 105 may record values ​​of the same item in different units, leading to data scaling issues. To address potential data scaling issues between different local sites 105, the model configuration module 310 in the master node 202 may determine definition information 312 indicating a range of scaled values ​​for the items in the medical data input to the initial medical validation model 230. The range of scaled values ​​may be, for example, a range from 0 to 1, or any other range. With the range of scaled values ​​of a particular item, the computing node 210 (i.e., the data preparation module 320) may process the historical medical data in the local training dataset 302 with values ​​of this item in the historical medical data values ​​within the range of scaled values. In this way, values ​​from the same range (i.e., the range of scaled values) may be determined for the same item across different local sites 105, which facilitates feature engineering in the initial medical validation model 230. This can be practical because federated learning assumes that the same features of the input are likely to follow the same distribution across different local sites.

[0075] In some embodiments, for an item in the medical data, its value may be calculated from two or more values ​​of other items in the test. For example, the ratio of free prostate specific antigen (FPSA) to total prostate specific antigen (TPSA), FPSA / TPSA, is calculated from the values ​​of FPSA and TPSA, and the anion gap is calculated based on the difference between the primary measured cations (sodium Na+ and potassium K+) and the primary measured anions (chloride Cl- and bicarbonate HCO3-) in serum. The computing node 210 (i.e., the data preparation module 320) may locally calculate the value of the item based on the original values ​​of the other items before scaling the other items to the scaled value range.

[0076] In some cases, other than the input item name, different local sites 105 may apply different criteria to divide the past medical data into different sets of validation categories. For example, a local site 105 may label the past medical data in the local training dataset 302 with two validation categories, one indicating that the past medical data is correct to be directly published and the other indicating that further validation is required. Another local site 105 may label the past medical data with three or more validation categories indicating a specific action that undergoes further validation. To enable the local model training module 340 to perform training in a supervised manner, in some embodiments, the model configuration module 310 in the master node 202 may determine the definition information 312 to indicate a unified validation category output from the initial medical validation model 230.

[0077] According to the definition information 312, the data preparation module 320 of the commuting node 210 may map the local validation category to the unified validation category. That is, the data preparation module 320 may apply the same labeling technique when updating or creating the labeling information in the local training dataset 302. In some examples, the data preparation module 320 may preserve the local validation category that is the same as the unified validation category (e.g., one with the same category name labeled to the past medical data in the local training dataset 302). In some examples, if the past medical data in the local training dataset 302 is labeled with a local validation category, but the definition information 312 indicates that this local validation category is to be finely divided and mapped to two or more unified validation categories, the data preparation module 320 may divide the past medical data and label them with two or more corresponding unified validation categories. In some other examples, the past medical data labeled with two or more local validation categories in the local training dataset 302 may be aggregated and labeled with one unified validation category to which the two or more local validation categories are mapped.

[0078] In some embodiments, the definition information may further indicate one or more unified red flag rules that prevent medical data from being input into the initial medical validation model. Since the medical validation procedure is to ensure that medical test reports with potential errors are not published, red flag rules may be set to ensure that medical test reports with serious or obvious errors are not erroneously determined to be correct by the medical validation model, taking into account the possibility of wrong diagnosis performed by the model. More specifically, medical data that meets the red flag rules may be directly passed to manual validation instead of being input into the medical validation model for automatic validation. Depending on different requirements and different regulations to be followed, different local sites 105 may apply different local red flag rules to prevent medical test reports that meet the local red flag rules from being passed to model-based automatic validation.

[0079] In training the early medical validation model 230, the master node 202 may configure one or more unified red flag rules in the definition information 312 to enable the computing node 210 to apply unified data filtering to medical data that may be input to the early medical validation model 230. The computing node 210 (e.g., the data preparation module 320 in the computing node 210) may process the local training dataset 302 by filtering out past medical data that meets one or more unified red flag rules. The unified red flag rules may define threshold-based criteria for an item in a medical test report. For example, the unified red flag may define that any medical test report with serum potassium higher than a threshold may not be published. If the value of this item in the past medical test report meets the threshold-based criteria, e.g., if the value is above the threshold or below the threshold (depending on how the criteria is set), the past medical test report may be filtered out from the local training dataset 302.

[0080] By applying the unified red flag rules to filter the local training dataset, the medical data that meets the unified red flag rules may not be used to train the initial medical validation model 230, which means that the model does not need to learn knowledge from the medical data that meets the unified red flag rules. In the next application stage where the final medical validation model 240 is applied, the medical data at the consumer site may also be filtered with the same unified red flag rules to ensure validation accuracy.

[0081] In some embodiments, the master node 202 may alternatively not configure uniform red flag rules for the local training data sets of the local sites 105. Thus, the initial medical validation model 230 may be trained without any restrictions on training data selection. As a result, the final medical validation model 240 is a rule-free model. In the next application stage of the final medical validation model 240, the consumer sites may apply their local red flag rules to determine which medical data can be passed to the final medical validation model 240 for automatic validation.

[0082] Above, several exemplary embodiments of model construction and data preparation have been described. After data preparation, each computing node 210 may generate a processed local training data set 322 for training. As shown, the master node 202 cooperates with the computing nodes 210 of the local sites 105 to perform a federated learning process to jointly train the initial medical validation model 230. During federated learning, the local model training module 340 in the computing node 210 may use the processed local training data set 322 to locally train the local medical validation model 230. The computing node 210 may apply a corresponding training algorithm to perform the training.

[0083] In some embodiments, the computing nodes 210 may generate parameter gradients 342 based on the processed local training dataset 322 and transmit the parameter gradients 342 to a training aggregation module 330 in the master node 202. The training aggregation module 330 may aggregate the parameter gradients received from multiple computing nodes 210 to determine parameter updates 332 for the parameters of the initial validation model 230. The parameter updates 332 may be transmitted to multiple computing nodes 210. In some embodiments, the parameter gradients 342 and / or the parameter updates 332 may be communicated over a secure channel between the computing nodes 210 and the master node 202 to prevent information leakage.

[0084] Using the parameter updates 332, the local model training module 340 in the computing node 210 may determine updated parameter values ​​of the initial validation model 230 to form an intermediate initial validation model and perform further training steps based on the intermediate initial validation model using the processed local training dataset 322. The exchange of parameter gradients and parameter updates between the master node 202 and the computing node 210 may be performed iteratively until a convergence condition of the federated learning process is reached. At that time, the training aggregation module 330 in the master node 202 may obtain a trained medical validation model 305 having trained parameter values ​​determined from the federated learning process.

[0085] In some embodiments, the master node 202 may determine the trained medical validation model 305 as the final medical validation model 240 ready to be distributed to the consumer site. In some embodiments, the master node 202 may perform a model validation procedure to verify whether the performance of the trained medical validation model 305 is suitable for distribution. To consider that the master node 202 may not have data to validate the model and different local sites 105 may have different validation criteria, the master node 202 may perform the model validation procedure in cooperation with the computing nodes 210 of the local sites 105.

[0086] Specifically, the model validation module 350 in the master node 202 may distribute the trained medical validation model 305 to the multiple computing nodes 210, for example, by sending trained parameter values ​​352 of the trained medical validation model 305 to the computing nodes 210. The local model validation module 360 ​​in the computing nodes 210 may use the processed local validation dataset 324 to determine performance metrics of the trained medical validation model 305. The processed local validation dataset 324 may be determined from the original local validation dataset 304 obtained from the database 220 in the corresponding local site 105. The processing of the local validation dataset 304 may be similar to the processing of the local training dataset 302, and the definition information 312 may also be utilized in the processing. During the local model validation process, the local model validation module 360 ​​in the computing node 210 may input the historical medical data in the processed local validation dataset 324 into the trained medical validation model 305 and determine whether the predicted validation results (indicating the validation category) output from the trained medical validation model 305 match the ground truth validation results in the processed local validation dataset 324.

[0087] Depending on the results of the local model validation process, the local model validation module 360 ​​in the computing node 210 may determine a performance metric indicative of the performance of the trained medical validation model 305. The performance metric may indicate, for example, an accuracy rate or a loss rate of a predicted validation result output from the trained medical validation model 305. Alternatively, or in addition, the performance metric may be determined based on a receiver operating characteristic (ROC) curve and / or an area under the curve (AUC). Other performance metrics may also be determined, and the scope of the disclosure is not limited in this respect. The local model validation module 360 ​​of each computing node 210 may send the performance metric as feedback 362 to the model validation module 350 of the master node 202.

[0088] After collecting the feedback of the performance metrics of the trained medical validation model 305 from the multiple computing nodes 210, the model validation module 350 in the master node 202 may determine the final medical validation model 240 based on the received feedback. In some embodiments, if the received performance metrics meet the model publication criteria, e.g., if the performance metrics from most or a certain number of computing nodes 210 indicate that the trained medical validation model 305 performs well in local medical validation, the model validation module 350 may determine that the trained medical validation model 305 may be distributed as the final medical validation model 240. In some embodiments, if the received performance metrics do not meet the model publication criteria, e.g., if the performance metrics from most or a certain number of computing nodes 210 indicate that the trained medical validation model 305 does not meet the performance when running locally, the model validation module 350 may determine that the trained medical validation model 305 may be further adjusted and thus a model fine-tuning process may be initiated to further update the parameter values ​​of the trained medical validation model 305.

[0089] In some embodiments, the model validation module 350 may distribute the trained medical validation model 305 as a final medical validation model to a computing node 210 from which a met performance metric (such as above a performance threshold) is received. The model validation module 350 may request to distribute the trained medical validation model 305 to other local sites to fine-tune the trained medical validation model 305 using their local training datasets.

[0090] In the above embodiment, the training and validation of one global medical validation model is described. In other embodiments, the master node 202 and the computing node 210 may jointly train multiple different medical validation models based on a federated learning process. The different medical validation models may be built with different processing algorithms (e.g., a model based on logistic regression and a model based on neural networks), trained with different training algorithms, and so on. Thus, the trained medical validation models from the federated learning process may have different performances despite being trained with the same local training dataset at the computing node 210.

[0091] By obtaining the different trained medical validation models and obtaining feedback from the computing nodes 210 indicative of their performance metrics, the model validation module 350 in the master node 202 may select one or more candidate medical validation models that meet the performance metrics of a particular consumer site (including local site 105 and other local sites such as local site 255). The computing nodes at the consumer sites may apply local datasets to further validate the performance of the candidate medical validation models and select appropriate models for use in local medical validation based on the performance metrics of the candidate medical validation models. The computing nodes at the consumer sites may fine-tune the candidate medical validation models using the local datasets as needed.

[0092] Example Process 4 is a flowchart of an example process for training a medical validation model implemented in a master node according to some embodiments of the present disclosure. Process 400 may be implemented in master node 202 of FIG. 2. For purposes of explanation, process 400 will be described with reference to FIG. 2.

[0093] In block 410, the master node 202 transmits definition information regarding the initial medical validation model to the multiple computing nodes 210. In block 420, the master node 202 performs a federated learning process with the multiple computing nodes 210 to jointly train the initial medical validation model using respective processed local training datasets available at the multiple computing nodes 210. The respective local training datasets are processed by the multiple computing nodes 210 based on the definition information. In block 430, the master node 202 determines a final medical validation model based on the results of the federated learning process.

[0094] In some embodiments, the master node 202 may distribute the final medical validation model to at least one of the plurality of computing nodes 210, or to at least one additional computing node, for use in medical validation.

[0095] In some embodiments, each local training data set may include historical medical data generated in a medical exam and labeling information indicating a local validation category of the historical medical data.

[0096] In some embodiments, the definition information indicates unified item names in the medical data input to the initial medical validation model and unified validation categories output from the initial medical validation model, the unified validation categories indicating a number of predefined validation actions to be performed on the medical data. In some embodiments, each local training dataset may be processed by mapping local item names used in the historical medical data to unified item names and mapping local validation categories to the unified validation categories.

[0097] In some embodiments, the definition information may further indicate a range of scaled values ​​for the items in the medical data input to the initial medical validation model. In some embodiments, each local training data set may be processed by mapping values ​​for the items in the historical medical data to values ​​within the range of scaled values.

[0098] In some embodiments, the definition information may further indicate uniform red flag rules for preventing medical data from being input into the initial medical validation model. In such cases, each local training data set may be processed by filtering out past medical data that meets the uniform red flag rules.

[0099] In some embodiments, the definition information may indicate an item in the medical data input to the initial medical validation model, and the value of the indicated item may not be available from the past medical data in the local training dataset. In such a case, the local training dataset is processed by filling in a predetermined value for the indicated item. In some embodiments, the predetermined value includes any one of the mean of the reference value range of the indicated item and the median of the available values ​​of the indicated item in the past medical data generated in other medical tests.

[0100] In some embodiments, to determine the final medical validation model, the master node 202 may obtain a trained medical validation model from the result of the federated learning process and distribute the trained medical validation model to the multiple computing nodes 210. The master node 202 may receive feedback from the multiple computing nodes 210 indicating the performance metrics of each of the trained medical validation models determined by the computing nodes 210 using the respective local validation data sets. The master node 202 may then determine the final medical validation model based on the received feedback. In some embodiments, in response to the respective performance metrics meeting the model publication criteria, the master node 202 may determine the trained medical validation model as the final medical validation model. In some embodiments, in response to the respective performance metrics not meeting the model publication criteria, the master node 202 may adjust the trained medical validation model to generate the final medical validation model.

[0101] In some embodiments, the master node 202 is communicatively connected to multiple computing nodes 210 in a star topology network.

[0102] 5 illustrates a flowchart of an example process 500 for training a medical validation model implemented on a computing node according to some embodiments of the present disclosure. The process 500 may be implemented on the computing node 210 of FIG. 2. For purposes of illustration, the process 500 will be described with reference to FIG. 2.

[0103] At block 510, the computing node 210 receives definition information for the early medical validation model from the master node 202. At block 520, the computing node 210 processes a local training data set based at least on the definition information. At block 530, the computing node 210 performs a federated learning process with the master node 202 and the at least one additional computing node to jointly train an early medical validation model using the processed local training data set.

[0104] In some embodiments, the computing node 210 may further receive from the master node 202 the final medical validation model determined from the federated learning process.

[0105] In some embodiments, the local training dataset includes historical medical data generated in a medical examination and labeling information indicative of a local validation category of the historical medical data.

[0106] In some embodiments, the definition information may indicate unified item names in the medical data input to the initial medical validation model and unified validation categories output from the initial medical validation model, the unified validation categories indicating a number of predefined validation actions to be performed on the medical data. In some embodiments, the computing node 210 may map local item names used in the historical medical data to the unified item names and map local validation categories to the unified validation categories.

[0107] In some embodiments, the definition information may further indicate a range of scaled values ​​for the item in the medical data input to the initial medical validation model. In some embodiments, the computing node 210 may map values ​​for the item in the historical medical data to values ​​within the range of scaled values.

[0108] In some embodiments, the definition information may further indicate uniform red flag rules to prevent medical data from being input into the initial medical validation model. In some embodiments, the computing node 210 may filter out historical medical data that meets the uniform red flag rules from the local training dataset.

[0109] In some embodiments, the definition information may indicate items in the medical data input to the initial medical validation model, where values ​​of the indicated items are not available from historical medical data generated in medical tests. In some embodiments, the computing node 210 may process the historical medical data by filling in the indicated items with predefined values.

[0110] In some embodiments, the predetermined value may include any one of the average of a reference value range for the indicated item and the median of available values ​​for the indicated item in historical medical data generated from other medical tests.

[0111] In some embodiments, the computing node 210 may further receive from the master node 202 a trained medical validation model determined from the results of the federated learning process. The computing node 210 may determine a performance metric of the trained medical validation model using the local validation dataset and send feedback to the master node 202 indicative of the determined performance metric.

[0112] In some embodiments, the computing node 210 may process the local validation data set based on the definition information and determine a performance metric using the processed local validation data set.

[0113] Exemplary Systems / Devices FIG. 6 illustrates a block diagram of an exemplary computing system / device 600 suitable for implementing exemplary embodiments of the present disclosure. The system / device 600 may be implemented or is implemented as the master node 202 or computing node 210 of FIG. 2. The system / device 600 may be a general-purpose computer, a physical computing device, or a portable electronic device, or may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communications network. The system / device 600 may be used to implement the process 400 of FIG. 4 and / or the process 500 of FIG. 5.

[0114] As shown, the system / device 600 includes a processor 601 capable of executing various processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 also stores data and the like required when the processor 601 executes the various processes, as necessary. The processor 601, the ROM 602, and the RAM 603 are connected to one another via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0115] The processor 601 may be of any type suitable for a local technology network and may include, by way of non-limiting examples, one or more of a general purpose computer, a special purpose computer, a microprocessor, a digital signal processor (DSP), a graphic processing unit (GPU), a co-processor, and a processor based on a multi-core processor architecture. The system / device 600 may have multiple processors, such as application specific integrated circuit chips that are slaved in time to a clock that synchronizes a main processor.

[0116] The multiple components in the system / device 600 are connected to an I / O interface 605 including an input unit 606 such as a keyboard or mouse, an output unit 607 including a display such as a cathode ray tube (CRT), liquid crystal display (LCD), and a speaker, etc., a storage unit 608 such as a disk, optical disk, etc., and a communication unit 609 such as a network card, modem, wireless transceiver, etc. The communication unit 609 enables the system / device 600 to exchange information / data with other devices via a communication network such as the Internet, various telecommunication networks, etc.

[0117] The above-mentioned methods and processes, such as process 400 and / or process 500, may also be executed by the processor 601. In some embodiments, the process 400 and / or process 500 may be implemented as a computer software program or computer program product tangibly contained in a computer-readable medium, such as the storage unit 608. In some embodiments, the computer program may be partially or fully loaded and / or embodied in the system / device 600 via the ROM 602 and / or the communication unit 609. The computer program includes computer-executable instructions that are executed by the associated processor 601. When the computer program is loaded into the RAM 603 and executed by the processor 601, one or more operations of the above-mentioned process 400 and / or process 500 may be performed. Alternatively, the processor 601 may be configured via any other suitable manner (e.g., with firmware) to execute the process 400 and / or process 500 in other embodiments.

[0118] Enumerated exemplary embodiments The embodiments of the present disclosure may be embodied in any of the forms described herein. For example, the exemplary embodiments listed below illustrate some of the structures, features, and functions of some of the aspects of the present disclosure disclosed herein.

[0119] In a first aspect, an exemplary embodiment of the present disclosure provides a computer-implemented method, the method including: sending, by a master node, definition information regarding an initial medical validation model to a plurality of computing nodes; performing, by the master node, a federated learning process together with the plurality of computing nodes to jointly train the initial medical validation model using respective processed local training datasets available at the plurality of computing nodes, where the respective local training datasets are processed by the plurality of computing nodes based on the definition information; and determining, by the master node, a final medical validation model based on a result of the federated learning process.

[0120] In some embodiments, the method further includes distributing, by the master node, the final medical validation model to at least one of the plurality of computing nodes or at least one further computing node for use in medical validation.

[0121] In some embodiments, each local training data set includes historical medical data generated in a medical examination and labeling information indicating a local validation category of the historical medical data.

[0122] In some embodiments, the definition information indicates unified item names in the medical data input to the initial medical validation model and unified validation categories output from the initial medical validation model, the unified validation categories indicating a number of predefined validation actions to be performed on the medical data. In some embodiments, each local training dataset is processed by mapping local item names used in the historical medical data to unified item names and mapping local validation categories to the unified validation categories.

[0123] In some embodiments, the definition information further indicates a range of scaled values ​​for the items in the medical data input to the initial medical validation model, hi some embodiments, each local training data set is processed by mapping values ​​for the items in the historical medical data to values ​​within the range of scaled values.

[0124] In some embodiments, the definition information further indicates uniform red flag rules for preventing medical data from being input into the initial medical validation model. In some embodiments, each local training data set is processed by filtering out historical medical data that meets the uniform red flag rules.

[0125] In some embodiments, the definition information indicates terms in the medical data input to the initial medical validation model, where values ​​of the indicated terms are not available from historical medical data in the local training dataset, hi some embodiments, the local training dataset is processed by inputting predetermined values ​​of the indicated terms.

[0126] In some embodiments, the predetermined value comprises one of the average of a reference value range for the indicated item and the median of available values ​​for the indicated item in historical medical data generated from other medical tests.

[0127] In some embodiments, determining a final medical validation model for at least one of the plurality of computing nodes includes: obtaining, by a master node, a trained medical validation model from a result of a federated learning process; distributing the trained medical validation model to the plurality of computing nodes; receiving feedback from the plurality of computing nodes, the feedback indicating respective performance metrics of the trained medical validation models determined by the computing nodes using their respective local validation datasets; and determining a final medical validation model based on the received feedback.

[0128] In some embodiments, the method wherein determining a final medical validation model based on the received feedback includes: determining a trained medical validation model as a final medical validation model in response to a respective performance metric satisfying a model publication criterion; and adjusting the trained medical validation model to generate the final medical validation model in response to the respective performance metric not satisfying the model publication criterion.

[0129] In some embodiments, the master node is communicatively connected to multiple computing nodes in a star topology network.

[0130] In a second aspect, an exemplary embodiment of the present disclosure provides a computer-implemented method, the method including: receiving, by a computing node, definition information from a master node regarding an early medical validation model, processing a local training dataset based on at least the definition information, and performing a federated learning process with the master node and at least one further computing node to jointly train an early medical validation model using the processed local training dataset.

[0131] In some embodiments, the method further includes receiving, by the computing nodes and from the master node, a final medical validation model determined from the federated learning process.

[0132] In some embodiments, the local training dataset includes historical medical data generated in a medical examination and labeling information indicative of a local validation category of the historical medical data.

[0133] In some embodiments, the definition information indicates unified item names in the medical data input to the initial medical validation model and unified validation categories output from the initial medical validation model, the unified validation categories indicating a number of predefined validation actions to be performed on the medical data. In some embodiments, processing the local training dataset includes mapping local item names used in the historical medical data to the unified item names and mapping the local validation categories to the unified validation categories.

[0134] In some embodiments, the definition information further indicates a range of scaled values ​​for the item in the medical data input to the initial medical validation model. In some embodiments, processing the local training dataset includes mapping values ​​for the item in the historical medical data to values ​​within the range of scaled values.

[0135] In some embodiments, the definition information further indicates uniform red flag rules for preventing the medical data from being input into the initial medical validation model. In some embodiments, processing the local training dataset includes filtering historical medical data that meets the uniform red flag rules from the local training dataset.

[0136] In some embodiments, the definition information indicates items in the medical data input to the initial medical validation model, and values ​​of the indicated items are not available from historical medical data generated in medical tests. In some embodiments, processing the local training dataset includes processing the historical medical data by filling in predetermined values ​​for the indicated items.

[0137] In some embodiments, the predetermined value comprises any one of the average of a reference value range for the indicated item and the median of available values ​​for the indicated item in historical medical data generated from other medical tests.

[0138] In some embodiments, the method further includes receiving, from the master node, a trained medical validation model determined from results of the federated learning process, determining a performance metric of the trained medical validation model using the local validation dataset, and sending feedback to the master node indicative of the determined performance metric.

[0139] In a third aspect, an exemplary embodiment of the present disclosure provides an electronic device comprising at least one processor and at least one memory including computer-readable instructions that, when executed by the at least one processor of the electronic device, cause the electronic device to perform the steps of the method in the first aspect described above.

[0140] In a fourth aspect, an exemplary embodiment of the present disclosure provides an electronic device comprising at least one processor and at least one memory including computer-readable instructions that, when executed by the at least one processor of the electronic device, cause the electronic device to perform the steps of the method in the second aspect described above.

[0141] In a fifth aspect, an exemplary embodiment of the present disclosure provides a computer program product comprising instructions which, when executed by a processor of an apparatus, cause the apparatus to perform any one of the steps of the method in the first aspect described above.

[0142] In a sixth aspect, an exemplary embodiment of the present disclosure provides a computer program product comprising instructions which, when executed by a processor of an apparatus, cause the apparatus to perform any one of the steps of the method in the second aspect above.

[0143] In a seventh aspect, an exemplary embodiment of the present disclosure provides a computer-readable medium including program instructions for causing an apparatus to perform at least the method in the first aspect above. The computer-readable medium may be a non-transitory computer-readable medium in some embodiments.

[0144] In an eighth aspect, an exemplary embodiment of the present disclosure provides a computer-readable medium including program instructions for causing an apparatus to perform at least the method of the second aspect above. The computer-readable medium may be a non-transitory computer-readable medium in some embodiments.

[0145] In general, various exemplary embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, and other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. Although various aspects of the exemplary embodiments of the present disclosure are illustrated and described using block diagrams, flowcharts, or some other graphical representations, it will be understood that the blocks, apparatus, systems, techniques, or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controller or other computing device, or some combination thereof.

[0146] The present disclosure also provides at least one computer program product tangibly stored in a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as those included in program modules, that execute on a target real or virtual processor device to perform the above-described methods / processes. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split among program modules as desired in various embodiments. The computer-executable instructions for the program modules may be executed in local or distributed devices. In distributed devices, the program modules may be located in both local and remote storage media.

[0147] The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More specific examples of the computer-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0148] Computer program code for carrying out the methods disclosed herein may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, performs the functions / operations specified in the flowcharts and / or block diagrams. The program code may be executed entirely on the computer, partially on the computer, as a stand-alone software package, partially on the computer, partially on a remote computer, or entirely on a remote computer or server. The program code may be distributed on specially programmed devices, which may be generally referred to herein as "modules". The software component parts of the modules may be written in any computer language and may be part of a monolithic code base or may be developed in more discrete code portions, as is typical in object-oriented computer languages. In addition, the modules may be distributed across multiple computer platforms, servers, terminals, mobile devices, etc. A given module may even be implemented such that the described functions are performed by separate processors and / or computing hardware platforms.

[0149] Although operations are shown in a particular order, this should not be understood as requiring such operations to be performed in the particular order shown, or in sequential order, or that all of the operations shown be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, while some specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure, but rather as descriptions of features that may be specific to certain embodiments. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination.

[0150] Although the present disclosure has been described in language specific to structural features and / or methodological acts, it is to be understood that the present disclosure, as defined by the appended claims, is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. transmitting, by a master node, definition information regarding an early medical validation model to a plurality of computing nodes; performing, by the master node, a federated learning process with the plurality of computing nodes to jointly train the initial medical validation model using respective processed local training datasets available at the plurality of computing nodes, wherein the respective local training datasets are processed by the plurality of computing nodes based on the definition information; determining, by the master node, a final medical validation model based on the results of the federated learning process; A computer-implemented method comprising:

2. distributing, by the master node, the final medical validation model to at least one of the plurality of computing nodes, or at least one further computing node, for use in medical validation; The method of claim 1 further comprising:

3. The method of claim 1 , wherein each of the local training data sets includes historical medical data generated in a medical examination and labeling information indicating a local validation category of the historical medical data.

4. The definition information indicates a unified item name in the medical data input to the initial medical verification model and a unified verification category output from the initial medical verification model, and the unified verification category indicates a plurality of predetermined verification actions to be performed on the medical data; each said local training data set is processed by mapping local item names used in said historical medical data to said unified item names and mapping said local validation categories to said unified validation categories; The method according to claim 3.

5. the definition information further indicates a scaled range of values ​​for items in the medical data that are input to the initial medical validation model; each said local training data set is processed by mapping values ​​of said items in said historical medical data to values ​​within said scaled value range; The method according to claim 3 or 4.

6. the definition information further indicates uniform red flag rules for preventing medical data from being input into the initial medical validation model; The method according to any of claims 2 to 5, wherein each said local training data set is processed by filtering historical medical data that satisfies said unified red flag rules.

7. the definition information indicates an item in medical data to be input to the initial medical validation model, a value of the indicated item being unavailable from historical medical data in a local training dataset; The method according to any of claims 2 to 6, wherein the local training data set is processed by inputting predefined values ​​for the indicated items.

8. 8. The method of claim 7, wherein the predetermined value comprises one of an average of a reference value range of the indicated item in past medical data generated in other medical tests and a median of available values ​​of the indicated item.

9. determining said final medical validation model, obtaining, by the master node, a trained medical validation model from the result of the federated learning process; and distributing the trained medical validation model to the plurality of computing nodes; and receiving feedback from the plurality of computing nodes, the feedback indicating a performance metric of each of the trained medical validation models determined by the computing nodes using a respective local validation dataset; determining the final medical validation model based on the received feedback; and The method according to any one of claims 1 to 8, comprising:

10. determining the final medical validation model based on the received feedback; determining the trained medical validation model as the final medical validation model in response to the respective performance metric satisfying a model publication criterion; adjusting the trained medical validation model to generate the final medical validation model in response to the respective performance metric not meeting the model publication criteria; 10. The method of claim 9, comprising:

11. The method according to any one of claims 1 to 10, wherein the master node is communicatively connected to the plurality of computing nodes in a star topology network.

12. receiving, by a computing node, definition information relating to an early medical validation model from a master node; processing a local training data set based on at least said definition information; performing a federated learning process with the master node and at least one further computing node to jointly train the initial medical validation model using the processed local training data set; A computer-implemented method comprising:

13. receiving, by the computing nodes, from the master node, a final medical validation model determined from the federated learning process; The method of claim 12 further comprising:

14. The method of claim 12 or 13, wherein the local training dataset comprises past medical data generated in a medical examination and labeling information indicative of a local validation category of the past medical data.

15. The definition information indicates a unified item name in the medical data input to the initial medical verification model and a unified verification category output from the initial medical verification model, and the unified verification category indicates a plurality of predetermined verification actions to be performed on the medical data; Processing the local training dataset comprises: mapping local item names used in the historical medical data to the unified item names and mapping the local validation categories to the unified validation categories; 15. The method of claim 14, comprising:

16. the definition information further indicates a scaled range of values ​​for items in the medical data that are input to the initial medical validation model; Processing the local training dataset comprises: mapping values ​​of the item in the historical medical data to values ​​within the scaled range of values; 16. The method of claim 14 or 15, comprising:

17. the definition information further indicates uniform red flag rules for preventing medical data from being input into the initial medical validation model; Processing the local training dataset comprises: filtering historical medical data from the local training dataset that meets the unified red flag rules; The method according to any one of claims 14 to 16, comprising:

18. the definition information indicates an item in medical data to be input to the initial medical validation model, a value of the indicated item being unavailable from historical medical data generated in a medical test; Processing the local training dataset comprises: processing said historical medical data by inputting predetermined values ​​into said indicated items; The method according to any one of claims 14 to 17, comprising:

19. 20. The method of claim 18, wherein the predetermined value comprises any one of an average of a reference value range of the indicated item in past medical data generated in other medical tests and a median of available values ​​of the indicated item.

20. receiving from the master node a trained medical validation model determined from an outcome of the federated learning process; determining a performance metric of the trained medical validation model using a local validation dataset; transmitting feedback to the master node indicative of the determined performance metric; The method according to any one of claims 12 to 19, comprising:

21. Determining the performance metric comprises: processing a local validation data set based on the definition information; determining the performance metric using the processed local validation data set; 21. The method of claim 20, comprising:

22. 1. An electronic device comprising: At least one processor; at least one memory containing computer readable instructions which, when executed by said at least one processor of said electronic device, cause said electronic device to perform the steps of the method according to any one of claims 1 to 11; An electronic device comprising:

23. 1. An electronic device comprising: At least one processor; at least one memory containing computer readable instructions which, when executed by said at least one processor of said electronic device, cause said electronic device to perform the steps of the method according to any one of claims 12 to 21; An electronic device comprising:

24. A computer program product comprising instructions which, when executed by a processor of a device, cause said device to perform the steps of the method according to any one of claims 1 to 11.

25. A computer program product comprising instructions which, when executed by a processor of an apparatus, cause said apparatus to perform the steps of the method according to any one of claims 12 to 21.

Citation Information

Patent Citations

  • Analyzer, analysis method, and analysis program

    JP2018190126A

  • Distributed machine learning system, apparatus, and method

    JP2019526851A

  • Grip force adjustment device and grip force adjustment system

    JP2020149505A

  • Automatized verification of medical data

    JP2021034056A

  • Federated learning system for training machine learning algorithms and maintaining patient privacy

    WO2021163213A1