Multi-center medical task model construction method based on model federated network

Abstract

The invention provides a multi-center medical task model construction method based on a model federated network, and the method comprises the steps: obtaining a model construction demand of a current medical center, and selecting a corresponding task model from the medical center as an original task model based on the model construction demand; selecting one or more external task models which are consistent with the task type of the original task model and are from other medical centers from the model federated network; performing feature attribution analysis on the original task model and each external task model to obtain model quantitative evaluation results of the original task model and each external task model; calculating the structural similarity between the original task model and each external task model in a graph matching mode; and based on the model quantitative evaluation result and the structural similarity, according to a preset mode, performing weighted fusion on neuron parameters of the neural network layer in the original task model and neuron parameters of the neural network layer corresponding to each external task model to obtain a target medical task model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and more specifically, to a medical task model construction technology in the field of artificial intelligence, and more specifically, to a multi-center medical task model construction method based on a model federated network. Background Art

[0002] With the explosive growth of medical data and the increasing demand for collaboration among multiple medical centers, how to effectively achieve collaboration and data sharing among multiple medical centers has become a key issue that needs to be solved urgently. Federated learning technology has emerged as an important tool to promote medical data sharing and task model training with its unique distributed learning framework. Federated learning protects data privacy and security and promotes the sharing and innovation of medical knowledge by allowing each medical center to jointly train a global model without directly exchanging original data. Specifically, federated learning technology uses a central server to coordinate the federated training process including multiple rounds of communication; in each round of federated training, the central server sends the global model parameters after federation aggregation to each participant (medical center); then, each participant retrains the local model based on local data and the received global model parameters, and transmits the retrained local model parameters back to the central server; finally, the central server performs federated aggregation based on the local model parameters received from each participant to update the global model. It can be seen that the distributed learning framework in federated learning technology not only alleviates the problem of data silos, but also opens up new ways for medical data processing across medical centers.

[0003] Although federated learning technology has made some progress in promoting collaboration and data sharing among multiple medical centers, there are still several problems with existing federated learning technology. The first is the data constraint problem: medical data is usually highly sensitive, complex, and has different quality. Medical centers are subject to strict legal and ethical restrictions on data sharing and use, which makes the effective training and updating of task models face huge challenges. Although federated learning technology avoids the direct exchange of raw medical data through a distributed learning framework, in actual operation, the data heterogeneity and quality differences among medical centers may still lead to insufficient generalization of the trained task model and the inability to fully adapt to the task requirements of different medical centers. The second is the task model update problem: in order to adapt the task model to the changes in the distribution of medical data and improve the performance and generalization of the task model, the task model needs to be retrained and updated. Task models in the medical field usually require a large amount of high-quality medical data and computing resources, and the cost of retraining and deploying task models is very high. The existing federated learning framework often requires frequent global communication and synchronization when updating the model and adapting to new tasks, which not only increases the communication overhead, but also prolongs the time cycle of task model updates, and limits its efficiency and feasibility in practical applications. The third is the problem of model adaptability and generalization ability: Due to the significant differences in data characteristics and distribution of each medical center, it is difficult for the task models of each medical center to achieve effective knowledge transfer and sharing between different centers. When faced with multi-task collaboration and model fusion tasks, there is often a lack of flexibility and adaptability, and it is impossible to dynamically adjust the task model structure and parameters of each medical center to adapt to different task requirements and scenario changes. Fourth, the problem of model security and privacy protection: Although federated learning technology improves data privacy through a distributed learning framework, the risk of information leakage during the task model parameter update process still exists. The existing security protocols and privacy protection technologies are still not perfect and sound in dealing with complex attacks and data leakage risks. Fifth, the problem of model selection and multi-task adaptation: When faced with complex requirements of multiple tasks and multiple scenarios, existing researchers respond by selecting high-quality task models obtained after federated learning training for fusion and collaboration. However, in the process of fusion and collaboration, there is often a lack of effective screening and optimization mechanisms, and it is impossible to give full play to the advantages of different task models to achieve the best synergy effect.

[0004] In summary, although the existing federated learning technology has made some progress in multi-medical center collaboration and data sharing, it still has significant problems and shortcomings in data constraints, task model updates, model adaptability and generalization capabilities, model security and privacy protection, model selection and multi-task adaptation.

[0005] It should be noted that this background technology is only used to introduce the relevant information of the present invention to help understand the technical solution of the present invention, but it does not mean that the relevant information is necessarily the prior art. If there is no evidence that the relevant information has been disclosed before the application date of the present invention, the relevant information shall not be regarded as the prior art. Summary of the invention

[0006] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a multi-center medical task model construction method based on a model federated network and a medical data classification method.

[0007] The objectives of the present invention are achieved through the following technical solutions.

[0008] According to a first aspect of the present invention, there is provided a method for constructing a multi-center medical task model based on a model federation network, wherein the model federation network is composed of all task models distributed in multiple medical centers, and each medical center includes one or more task models, and the model federation network includes task models of multiple different task types, wherein the task model includes a multi-layer neural network layer, and each neural network layer includes a plurality of neurons, and the method includes: responding to the model construction requirements of the medical center and performing the following steps in the medical center: obtaining the model construction requirements of the current medical center, and selecting the corresponding task model from the medical center as the original task model based on the model construction requirements; selecting one or more task models from other medical centers that are consistent with the task type of the original task model from the model federation network External task model; performing feature attribution analysis on the original task model and each external task model respectively to obtain model quantitative evaluation results of the original task model and each external task model, wherein the model quantitative evaluation results represent the key features related to the task that can be identified by the model and the neurons related to the key features; calculating the structural similarity between the original task model and each external task model in a graph matching manner, wherein the structural similarity represents the similarity of each neuron in the corresponding neural network layer between the original task model and each external task model; based on the model quantitative evaluation results and the structural similarity, weightedly fusing the neuron parameters of the neural network layer in the original task model with the neuron parameters of the corresponding neural network layer of each external task model in a preset manner to obtain the target medical task model.

[0009] In some embodiments of the present invention, the preset method is: based on structural similarity, neurons whose similarity with corresponding neurons in the original task model is greater than or equal to a preset threshold are selected from each external task model as fusion neurons, and based on the model quantitative evaluation results, neurons related to key features that are not selected in each external task model are also used as fusion neurons; all fusion neuron parameters are weightedly fused with the corresponding neuron parameters in the original task model to obtain the target medical task model.

[0010] In some embodiments of the present invention, each medical center is configured with multiple intelligent agents for performing incremental model fusion. The method also includes: responding to the incremental tasks of the medical center, using the target medical task model constructed by the medical center as the initial task model, and using the intelligent agent to obtain the task model corresponding to the incremental task from other medical centers to perform model incremental fusion processing on the initial task model to obtain a fused medical task model.

[0011] In some embodiments of the present invention, the incremental task is split into multiple subtasks and assigned to different agents for execution. Each subtask corresponds to one or more agents. Each agent executes its assigned subtask and passes the task execution results to other agents associated with the subtask executed by the agent, so that other agents execute their own subtasks in combination with the received task execution results and continue to pass their own task execution results to the next agent until all agents complete their subtasks to obtain a fused medical task model; wherein the incremental task is split into a model capability evaluation subtask, a model routing scheduling subtask and a model incremental fusion subtask; the model capability evaluation subtask is used to evaluate the ability of the task model to perform different tasks to determine the target fusion task model; the model routing scheduling subtask is used to determine the fusion order of the target fusion task model; the model incremental fusion subtask is used to incrementally fuse the target fusion task model with the initial medical task model.

[0012] In some embodiments of the present invention, the agent that performs the model capability evaluation subtask performs the model capability evaluation subtask in the following manner: based on the incremental task of the current medical center, multiple task models from other medical centers corresponding to the incremental task are selected from the model federation network as task models to be fused; each task model to be fused is scored according to a preset scoring method, and a task model to be fused with a score greater than or equal to a preset score is selected from the scoring results as the target fusion task model for incremental fusion; wherein the preset scoring method is:

[0013]

[0014] Among them, K HRepresents the kernel matrix, which is used to characterize the sensitivity of the task model to data distribution; N A represents the number of neurons in the task model; d H (c N ,c1) represents the Hamming distance of the overlap between the activation of the Nth neuron and the first neuron node in the task model, c N represents the activation of the Nth neuron, and c1 represents the activation of the first neuron.

[0015] In some embodiments of the present invention, the agent executing the model routing scheduling subtask executes the model routing scheduling subtask in the following manner: determining the fusion order of the target fusion task model when performing incremental fusion according to a preset scheduling method, wherein the preset scheduling method is:

[0016]

[0017] Among them, C total represents the total cost of incremental fusion according to the fusion order, x ij represents the decision variable, x ij =1 means that the i-th task model is assigned the corresponding fusion scheme for incremental fusion with the j-th task model; c ij Represents the execution decision variable x ij The cost consumed by the corresponding fusion solution.

[0018] In some embodiments of the present invention, the agent executing the model incremental fusion subtask executes the model incremental fusion subtask in the following manner: performing neuron scoring on each target fusion task model according to a preset neuron scoring method to obtain a score of each neuron in each target fusion model, wherein the preset neuron scoring method is:

[0019]

[0020] Among them, APE i,j represents the activation probability entropy of the jth neuron in the i-th neural network layer of the task model, l represents the total number of task categories, It represents the probability that the jth neuron in the ith neural network layer is activated under the kth type of task; the neurons with a score greater than or equal to the preset neuron score in each target fusion task model are used as incremental neurons, and the parameters of the incremental neurons in each target fusion task model are weightedly fused with the corresponding neuron parameters in the initial medical task model in the fusion order to obtain the fused medical task model.

[0021] According to a second aspect of the present invention, a medical data classification method is provided, the method comprising: step S1, obtaining medical data to be processed; step S2, classifying the medical data to be processed using the medical task model obtained by the method described in the first aspect of the present invention.

[0022] Compared with the prior art, the advantages of the present invention are: (1) without the need to retrain the task model with the help of medical data, the task models of other medical centers are selected to carry out cross-medical center model fusion to achieve knowledge transfer and sharing, thereby reducing time cost and economic burden while updating the task model, thereby improving the model's adaptability and generalization ability; (2) by assigning different roles to intelligent agents, they collaborate in the planning and execution of different tasks and services, so that medical centers can adapt to complex tasks by selecting, combining, and fusing different task models, thereby building a medical task model that can cope with complex requirements of multiple tasks and multiple scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0024] Figure 1 The present invention is a flowchart of a method for constructing a multi-center medical task model based on a model federation network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0026] As mentioned in the background technology section, although the existing federated learning technology has made some progress in multi-medical center collaboration and data sharing, it still has significant problems and shortcomings in data constraints, task model updates, model adaptability and generalization capabilities, model security and privacy protection, model selection and multi-task adaptation, etc.

[0027] In order to solve the shortcomings of the existing technology, the inventors analyzed the existing technology to seek solutions. It can be seen from the analysis that under the sensitivity and compliance constraints of medical data, the difficulty of retraining and adjusting the task model is particularly prominent. In practical applications, there are multiple trained task models in medical centers, which have consumed a lot of resources and time in the early stages of development; however, with the changes in medical task requirements and the continuous emergence of new medical data, the original task models often cannot meet the latest medical task requirements and research goals. In this case, retraining the task model becomes an inevitable choice, but this brings new difficulties. The cost of retraining the task model is high, especially in the medical field. Data annotation and model training require a lot of human resources and computing resources, and each adjustment and update of the task model requires recollection, collation and annotation of a large amount of data, which not only increases the time cost and economic burden, but also affects the timely application and promotion of the task model. Even if you are willing to pay a high cost to update the task model, the high sensitivity and privacy protection requirements of medical data are strictly restricted in data sharing and use, and any form of data leakage will bring serious legal and ethical issues. Therefore, in the collaboration of multiple medical centers, it is necessary to update and optimize the task model without directly exchanging data, and in the process of updating and optimizing, overcome problems such as model adaptability and generalization ability, model security and privacy protection, model selection and multi-task adaptation, etc.

[0028] Based on the analysis of the existing technology, the inventor proposed a new task model construction scheme, in which the task model is not retrained with the help of medical data, but the task models of other medical centers are selected to carry out cross-medical center model fusion to achieve knowledge transfer and sharing, while reducing time and economic costs to update the task model, thereby improving the model's adaptability and generalization ability; further, during the incremental fusion processing of the model, the intelligent agent is used to perform task model selection and multi-task model fusion to obtain a task model that can cope with complex requirements of multiple tasks and multiple scenarios. At the same time, in the model construction scheme, efficient security protocols and privacy protection technologies are used to ensure the security of medical data and model parameters.

[0029] In summary, if Figure 1As shown, the present invention provides a method for constructing a multi-center medical task model based on a model federation network, wherein the model federation network is composed of all task models distributed in multiple medical centers, and each medical center includes one or more task models, and the model federation network includes task models of multiple different task types, wherein the task model includes a multi-layer neural network layer, and each neural network layer includes a plurality of neurons. The method includes responding to the model construction requirements of the medical center and performing the following steps in the medical center: obtaining the model construction requirements of the current medical center, and selecting a corresponding task model from the medical center as the original task model based on the model construction requirements; selecting one or more external task models from other medical centers that are consistent with the task type of the original task model from the model federation network; service model; perform feature attribution analysis on the original task model and each external task model respectively to obtain model quantitative evaluation results of the original task model and each external task model, the model quantitative evaluation results characterize the key features related to the task and the neurons related to the key features that the model can identify; calculate the structural similarity between the original task model and each external task model in a graph matching manner, the structural similarity characterizes the similarity of each neuron in the corresponding neural network layer between the original task model and each external task model; based on the model quantitative evaluation results and structural similarity, perform weighted fusion of the neuron parameters of the neural network layer in the original task model and the neuron parameters of the corresponding neural network layer of each external task model in a preset manner to obtain the target medical task model.

[0030] In order to better understand the present invention, each step in the method for constructing a medical task model is described in detail below in conjunction with specific embodiments.

[0031] Before introducing the medical task model construction method, let's first briefly introduce the two types of tasks included in the method, one is the model construction task, and the other is the model incremental fusion task. Among them, the model construction task is to select the corresponding task model from the medical center as the original task model based on the model construction requirements of the medical center, and select one or more external task models from other medical centers that are consistent with the task type of the original task model from the model federation network to update the original task model to build the target medical task model. The model incremental fusion task responds to the incremental task of the medical center and uses the intelligent agent to obtain the task model corresponding to the incremental task from other medical centers to perform model incremental fusion processing on the initial task model. The model construction task and the model incremental fusion task are described in detail below.

[0032] 1. Model building tasks

[0033] The model building task is to select the corresponding task model from the medical center as the original task model based on the model building requirements of the medical center, and select one or more external task models from other medical centers with the same task type as the original task model from the model federation network to update the original task model to build the target medical task model. The steps for executing the model building task are described in detail below.

[0034] First, the model building requirements of the current medical center are obtained, and based on the model building requirements, the corresponding task model is selected from the medical center as the original task model.

[0035] Then, one or more external task models from other medical centers that are consistent with the task type of the original task model are selected from the model federation network. It should be noted that under the sensitivity and compliance constraints of medical data, collecting medical data and annotating it for model retraining obviously requires a lot of human resources and computing resources. In order to reduce costs, the present invention updates the original task model by selecting an external task model that is consistent with the task type of the original task model from other medical centers to perform cross-medical center fusion, which can not only maximize the use of the information of the existing task model, but also improve the model adaptability and generalization ability of the original task model, and the transfer of model knowledge will not expose personal information or sensitive information. It should also be noted that different medical centers can communicate with each other, and when the task model is transferred across medical centers during the communication process, privacy computing technologies such as differential privacy, homomorphic encryption, and multi-party secure computing can be used to provide information security for model transmission.

[0036] Then, feature attribution analysis is performed on the original task model and each external task model to obtain model quantitative evaluation results of the original task model and each external task model, wherein the model quantitative evaluation results characterize the key features related to the task and the neurons related to the key features that the model can recognize. The feature attribution analysis is described in detail below.

[0037] The present invention adopts a feature attribution reverse analysis method based on information bottleneck to identify the key features learned by the task model by analyzing the behavior of the task model when processing different medical data. This process does not rely on gradient information or any form of interpretability indication during the task model training process, but directly starts from the representation of the task model itself, combined with the original input medical data, to try to understand the dependence and sensitivity of the task model on certain specific input features. In addition, by quantitatively evaluating the input space information of different task models, in-depth insights into how the task model understands and processes the input data can be obtained. This evaluation can be based on the response of the task model output to the input medical data, for example, by comparing the response differences of the task model to similar medical data, thereby quantifying the sensitivity and distinguishing ability of the task model to the input medical data features. It should be noted that the feature attribution reverse analysis method aims to improve the recognition accuracy of the task model for the input medical data features through fine internal analysis, and to ensure the universality of the feature attribution reverse analysis method when performing cross-task model metric analysis. It should also be noted that the feature attribution reverse analysis method evaluates the responsiveness of the task model to specific features or data patterns from a micro scale. By comparing the processing effects of different task models on the same medical data set or different medical data sets, the generalization ability and adaptability of different task models to new data are evaluated. The key task-related features and neurons related to the key features in each task model are analyzed by obtaining the model quantitative evaluation results of each task model.

[0038] Information bottleneck is a method to understand and analyze the process of information compression and feature extraction. Its core goal is to find a balance between information compression and retention of key information, that is, to find a compressed representation of input data that retains as much information as possible related to a specific target variable while removing as much information as possible that is irrelevant to the target variable. Based on the information bottleneck principle, the present invention inserts an information bottleneck between the input layer of the task model and the first hidden layer to limit the flow of information that is irrelevant to the target prediction, thereby achieving feature attribution analysis to obtain the model quantitative evaluation results of each external task model. This processing method can expand the application scope of the information bottleneck from within the task model to after the task model input, thereby directly controlling the task model's processing method for input medical data.

[0039] The specific processing methods include generative model application and information bottleneck insertion. In which, generative model application: first, the Z calculated by the generative model is * , Z *represents a specific bottleneck variable obtained through the optimization process, which contains the best feature representation that maximizes information retention and minimizes redundancy; after the information bottleneck is inserted into the input layer, the neural network is learned by replacing the information bottleneck with predicted prior knowledge. * The process is: Z * represents the bottleneck variable; D(·) represents the distance metric function; f(Z G ) represents Z G The hidden features obtained after processing the first half of the task model, Z G Represents the actual intermediate features calculated; Z represents the input features. Information bottleneck insertion: By adding noise to the hidden layer, the optimization function is constructed using the information bottleneck theory to remove as much information in the hidden layer that is irrelevant to the prediction information as possible. Among them, the optimization function is expressed as: minI(R; Z * )-βI(Z * ; Y), I (;) represents mutual information, Y represents the prediction target, β is a balance parameter used to control the degree of information elimination, and R represents the hidden layer. It should be noted that the application of the generative model can not only accurately identify the prediction information in the input medical data features, but also be applicable to different task models, and the application of the information bottleneck theory is moved forward to the input of the task model. It can analyze the key features learned by the task model by controlling the input medical data information flow without relying on the internal gradient information of the task model. It should be noted that the information bottleneck insertion can help achieve more scientific model selection and weight allocation by clarifying the contribution of each task model to the prediction results and the influence of key input features, that is, more important features and neurons related to these features will be given higher weights to guide the fusion of multiple task models. In general, the feature attribution reverse analysis method based on information bottleneck can identify important features and neurons related to these important features by clarifying the contribution of each task model to the prediction results and the influence of key input features. This not only helps to improve the credibility and transparency of the fusion model in sensitive areas, but also optimizes the fusion strategy and strengthens the model's ability to learn key features, thereby improving the generalization ability of the fusion model. At the same time, the information revealed by feature attribution helps to identify and adjust the model's potential bias and unfair factors, further ensuring the fairness of the model fusion process.

[0040] Then, the structural similarity between the original task model and each external task model is calculated in a graph matching manner, wherein the structural similarity represents the similarity of each neuron in the corresponding neural network layer between the original task model and each external task model. The graph matching manner is described below.

[0041] The present invention models the task model fusion problem of the same task type as a graph matching problem (graph matching is used to match the neurons corresponding to each layer of the task model one by one, and the corresponding results are used to guide the task model to be fused and updated), and considers the inherent second-order properties in the problem. The neurons in the task model are regarded as nodes in graph matching, and the model weights connecting the neurons are regarded as edges in graph matching. The goal of graph matching is to map and match the nodes in the two graphs so as to maximize the similarity between points and points and the similarity between edges after matching. Graph matching considers both the first-order similarity and the second-order similarity in the graph, especially the structural information in the graph. Therefore, it is very natural and easy to solve to model model fusion as a graph matching problem.

[0042] Taking task model 1 and task model 2, which are composed of two four-layer fully connected networks, as examples, we will explain how to model model fusion as a graph matching problem. The permutation matrices P0, P1, P2, and P3 are used to represent the correspondence between the neurons in each layer of the two task models. The total permutation matrices P0, P1, P2, and P3 are placed along the diagonal, and the remaining elements are 0. The meaning of these permutation matrices is that the neurons in task model 1 can be aligned with the neurons in task model 2 after matrix multiplication with the permutation matrices. Due to the nature of the task model fusion problem of the same task type, the neurons in the first and last layers do not need to be matched and are already aligned, so P0 and P3 are both unit matrices. At the same time, because neurons in different layers cannot be matched across layers, the overall matching matrix is ​​in a stepped shape. This feature provides a solution to the problem of large-scale fusion of task models of the same task type. Among them, the second-order similarity measure of the graph can be calculated using methods such as Gaussian kernel, mutual information, information entropy, and kernel function on the graph. In this way, the task model fusion problem of the same task type can be modeled as a graph matching problem and optimized using a new objective function, which is expressed as follows:

[0043]

[0044] Among them, P represents the neuron permutation matrix, which is used to arrange the neurons between the two hidden layers. P is not limited to a single matrix, but represents a set of permutation matrices P0, P1, P2, P3. P0 and P3 usually represent identity matrices, that is, the neurons in the input layer and the output layer are not arranged. [a,b,f,k] Represents a four-dimensional similarity tensor, which is used to measure the similarity between edges (a,f) and (b,k). In graph matching problems, similarity usually reflects the similarity between weight matrix elements in task matching model fusion problems of the same task type. Σrepresents the sum of all neurons, and I represents the identity matrix, which is usually used to ensure that the neurons in the input and output layers are not permuted. The constraint ensures that the neurons in the same layer correspond one to one. The objective function aims to maximize the similarity tensor K by permuting the matrix P [a,b,f,k] The weighted sum of is used to find the best arrangement of neurons in the hidden layer to solve the model fusion or matching problem. The neurons in the input and output layers remain unchanged, while the arrangement of neurons between hidden layers is constrained to ensure the unique correspondence of each neuron.

[0045] Although the graph matching method can calculate the structural similarity between the original task model and each external task model, in the calculation process, a common task model generally has 512 / 1024 neurons per layer. For a task model with ten neural network layers, the total number of neurons will be tens of thousands. Corresponding to the graph matching problem, it is equivalent to having tens of thousands of nodes in a graph. However, the number of nodes in a general graph matching task is less than 100. At this time, the scale of the structural similarity solution problem between task models is very different. To address this problem, an iterative memory-efficient slice scanning algorithm can be used to solve it. The algorithm is based on the key observation of the intrinsic sparsity characteristics of the problem, that is, the previously mentioned feature that neuron matching should be limited to the same layer. A three-layer wide window is used to continuously scan the task model. In each iteration, the permutation matrices of the upper and lower layers are kept unchanged, and the permutation matrix of the middle layer is updated based on this. At the same time, in order to more efficiently solve the structural similarity between multiple task models, the extension of multi-graph matching is supported, and the concept of model consistency in multi-graph matching is considered when matching neurons. Afterwards, classic combinatorial optimization algorithms such as the Hungarian algorithm or the Hopcroft-Karp algorithm are used to solve the graph matching problem, namely the neuron alignment problem, to obtain a suitable arrangement matrix for neuron sorting transformation, and to fuse multiple task models after neuron alignment.

[0046] Finally, based on the model quantitative evaluation results and structural similarity, the neuron parameters of the neural network layer in the original task model and the neuron parameters of the corresponding neural network layer of each external task model are weighted and fused in a preset manner to obtain the target medical task model. According to one embodiment of the present invention, the preset manner is: based on the structural similarity, neurons whose similarity with the corresponding neurons in the original task model is greater than or equal to the preset threshold are selected from each external task model as fusion neurons, and based on the model quantitative evaluation results, neurons related to key features that are not selected in each external task model are also used as fusion neurons; all fusion neuron parameters are weighted and fused with the corresponding neuron parameters in the original task model to obtain the target medical task model. It should be noted that weighted fusion refers to assigning different weights to the parameters of each neuron and then performing weighted summation, and assigning the value obtained after the weighted summation as a new parameter to the neuron, wherein the more important the neuron is in the weighting process, the greater the weight, and the specific weight value is adjusted according to actual needs, and the present invention does not impose special restrictions. It should also be noted that the structural similarity and model quantitative evaluation results help reveal the dependency and importance of the task model on the input medical data features, and can be used to guide the optimization process of the permutation matrix to ensure that not only the neurons of the task model are aligned, but also the most important features are prioritized and retained. For example, if a neuron is found to be critical to the prediction task through feature attribution, then in the process of graph matching, this match can be preferred by adjusting the objective function to ensure that the key neurons are retained and play a role in the fused model. In this way, feature attribution not only improves the interpretability of model fusion, but also provides a more refined and goal-oriented optimization goal for fusion, thereby improving the performance and generalization ability of the fusion model.

[0047] 2. Model Incremental Fusion Task

[0048] The model incremental fusion task responds to the incremental task of the medical center and uses the target medical task model constructed by the medical center as the initial task model. The intelligent agent obtains the task model corresponding to the incremental task from other medical centers to perform model incremental fusion processing on the initial task model. It should be noted that the medical task model after the model incremental fusion processing has the ability to handle other medical tasks compared to the initial medical task model. For example, the initial medical task model can only distinguish between myopia and non-myopia. The medical task model obtained after incremental processing can distinguish between glaucoma and cataracts on this basis. The following is a detailed description of the process of executing the model incremental fusion task.

[0049] According to one embodiment of the present invention, in the process of executing a model incremental fusion task, the incremental task is split into multiple subtasks and then assigned to different agents for execution. Each subtask corresponds to one or more agents. Each agent executes its assigned subtask and passes the task execution result to other agents associated with the subtask executed by the agent, so that other agents execute their own subtasks in combination with the received task execution results and continue to pass their own task execution results to the next agent until all agents complete their subtasks to obtain a fused medical task model; wherein the incremental task is split into a model capability evaluation subtask, a model routing scheduling subtask and a model incremental fusion subtask. The model capability evaluation subtask is used to evaluate the ability of the task model to perform different tasks to determine the target fusion task model, and the model routing scheduling subtask is used to determine the fusion order of the target fusion task model; the model incremental fusion subtask is used to incrementally fuse the target fusion task model with the initial medical task model.

[0050] It should be noted that in the process of assigning subtasks to various agents, it is necessary to effectively manage the information flow and decision conflicts between different agents, coordinate the planning and constraint optimization of agent behaviors, and evaluate and adjust the task allocation performance and task adaptation according to the reward and punishment mechanism. The state value function measures the cumulative reward that the agent expects to obtain when starting in state s and following the strategy π, expressed as:

[0051]

[0052] V π (s) represents the total return expected from executing actions according to the strategy in state s under strategy π, The expectation of the weighted average return of all possible paths; represents the cumulative sum of rewards starting from time step t, and uses the penalty factor γ k To reduce the impact weight of future returns, its value is between 0 and 1; it is used to indicate the relative importance of future returns relative to current returns. The closer the value is to 1, the higher the weight of future returns, and the closer the value is to 0, the lower the weight of future returns; R t+k+1 It represents the immediate reward of the state s at the current time t.

[0053] The action value function measures the cumulative reward that the agent expects to obtain when taking action a in state s and then following the policy π, expressed as:

[0054]

[0055] Q π(s,a) represents the action-value function (Action-ValueFunction) of state s and action a under strategy π. This value function represents the total return expected from taking action a in a given state s and then executing actions according to strategy π; The expectation represents the weighted average return of all possible paths after taking action a from state s under policy π; represents the cumulative sum of rewards starting from time step t, and uses the penalty factor γ k To reduce the impact weight of future returns, its value is between 0 and 1; R t+k+1 It indicates the immediate reward of the current state s and action a at time t.

[0056] By evaluating and adjusting the task allocation performance and task adaptation through the above-mentioned reward and punishment mechanism, not only can the efficiency and accuracy of each intelligent agent in processing different subtasks be improved, but also the subtask allocation can obtain the maximum benefit.

[0057] According to one embodiment of the present invention, the agent that performs the model capability evaluation subtask performs the model capability evaluation subtask in the following manner: based on the incremental task of the current medical center, multiple task models from other medical centers corresponding to the incremental task are selected from the model federation network as task models to be fused; each task model to be fused is scored according to a preset scoring method, and a task model to be fused with a score greater than or equal to a preset score is selected from the scoring results as a target fusion task model for incremental fusion; wherein the preset scoring method is:

[0058]

[0059] Among them, K H Represents the kernel matrix, which is used to characterize the sensitivity of the task model to data distribution; N A represents the number of neurons in the task model; d H (c N ,c1) represents the Hamming distance of the overlap between the activation of the Nth neuron and the first neuron node in the task model, c NRepresents the activation status of the Nth neuron, and c1 represents the activation status of the first neuron. It should be noted that for the input data of model training, the input data can be simply divided into two categories: activation and inactivation of a neuron. Then, a segmentation surface will be generated for the input data, and the segmentation surfaces generated by each neuron can be superimposed on each other, and the final model will achieve complex prediction capabilities. In addition, there is also a superposition relationship between different neural network layers of the model, and the model has a more powerful nonlinear fitting ability. It is based on this observation that the present invention proposes to evaluate the sensitivity of the model to data distribution by analyzing the overlap of activations between neurons in the model. It should be noted that the kernel matrix K H Each element of is calculated by the Hamming distance between the activations of different data nodes in the task model (Hamming distance is used to represent the difference between two neurons, and the larger the value, the greater the difference between neurons). A high score of this matrix means good prediction performance, which can be used to evaluate the adaptability of different task models or network structures to data distribution, thereby helping to select the optimal task model or network structure. It should also be noted that since the preset scoring method does not require actual reasoning on the task model, the scoring process is very fast.

[0060] According to one embodiment of the present invention, the agent executing the model routing scheduling subtask executes the model routing scheduling subtask in the following manner: determining the fusion order of the target fusion task model when performing incremental fusion according to a preset scheduling method, wherein the preset scheduling method is:

[0061]

[0062] Among them, C total represents the total cost of incremental fusion according to the fusion order, x ij represents the decision variable, x ij =1 means that the i-th task model is assigned the corresponding fusion scheme for incremental fusion with the j-th task model; c ij Represents the execution decision variable x ij The cost consumed by the corresponding fusion solution. It should be noted that the fusion order can effectively route and schedule the target fusion task model, optimize resource usage, reduce completion time, and improve the fusion efficiency and performance of the target fusion task model.

[0063] According to one embodiment of the present invention, the agent executing the model incremental fusion subtask executes the model incremental fusion subtask in the following manner: performing neuron scoring on each target fusion task model according to a preset neuron scoring method to obtain a score of each neuron in each target fusion model, wherein the preset neuron scoring method is:

[0064]

[0065] Among them, APE i,j represents the activation probability entropy of the jth neuron in the i-th neural network layer of the task model, l represents the total number of task categories, It represents the probability that the jth neuron in the i-th neural network layer is activated under the k-th task; the neurons with a score greater than or equal to the preset neuron score in each target fusion task model are used as incremental neurons, and the parameters of the incremental neurons in each target fusion task model are weightedly fused with the corresponding neuron parameters in the initial medical task model in the fusion order to obtain the fused medical task model. It should be noted that neurons with lower APE scores are called task-specific neurons because they show a preference for activation of a certain category, while reducing the probability of activation of other categories.

[0066] Based on the medical task construction method described in the aforementioned embodiment, the medical task model constructed therein can be used for medical data classification. Based on this, the present invention also provides a medical data classification method, which includes: step S1, obtaining medical data to be processed; step S2, using the medical task model obtained in the aforementioned embodiment to classify the medical data to be processed.

[0067] Compared with the prior art, the beneficial effects of the present invention are: (1) without the need to retrain the task model with the help of medical data, the task models of other medical centers are selected to carry out cross-medical center model fusion to achieve knowledge transfer and sharing, thereby reducing time cost and economic burden while updating the task model, thereby improving the model's adaptability and generalization ability; (2) by giving intelligent agents different roles, they collaborate in the planning and execution of different tasks and services, so that medical centers can adapt to complex tasks by selecting, combining, and fusing different task models, thereby building a medical task model that can cope with complex requirements of multiple tasks and multiple scenarios.

[0068] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0069] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0070] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0071] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for constructing a multi-center medical task model based on a model federation network, wherein the model federation network is composed of all task models distributed in multiple medical centers, and each medical center includes one or more task models, and the model federation network includes task models of multiple different task types, wherein: The task model includes multiple neural network layers, each neural network layer includes multiple neurons, and the method includes: responding to the model building requirements of the medical center and performing the following steps in the medical center: Obtain the model building requirements of the current medical center, and select a corresponding task model from the medical center as the original task model based on the model building requirements; Select one or more external task models from other medical centers that are consistent with the task type of the original task model from the model federation network; Performing feature attribution analysis on the original task model and each external task model respectively to obtain model quantitative evaluation results of the original task model and each external task model, wherein the model quantitative evaluation results represent the key features related to the task and neurons related to the key features that can be recognized by the model; Calculating the structural similarity between the original task model and each external task model in a graph matching manner, wherein the structural similarity represents the similarity of each neuron in the corresponding neural network layer between the original task model and each external task model; Based on the quantitative evaluation results and structural similarity of the model, the neuron parameters of the neural network layer in the original task model are weightedly fused with the neuron parameters of the corresponding neural network layers of each external task model in a preset manner to obtain the target medical task model.

2. The method according to claim 1, characterized in that The preset method is: Based on the structural similarity, neurons whose similarity with the corresponding neurons in the original task model is greater than or equal to a preset threshold are selected from each external task model as fusion neurons, and based on the model quantitative evaluation results, neurons related to key features that are not selected in each external task model are also used as fusion neurons; All fused neuron parameters are weightedly fused with the corresponding neuron parameters in the original task model to obtain the target medical task model.

3. The method according to claim 2, characterized in that Each medical center is configured with multiple intelligent agents for performing incremental model fusion. The method also includes: responding to the incremental tasks of the medical center, taking the target medical task model constructed by the medical center as the initial task model, and using the intelligent agent to obtain the task model corresponding to the incremental task from other medical centers to perform model incremental fusion processing on the initial task model to obtain a fused medical task model.

4. The method according to claim 3, characterized in that The incremental task is split into multiple subtasks and assigned to different agents for execution. Each subtask corresponds to one or more agents. Each agent executes its assigned subtask and passes the task execution results to other agents associated with the subtask executed by the agent, so that other agents execute their own subtasks in combination with the received task execution results and continue to pass their own task execution results to the next agent until all agents complete their subtasks to obtain a fused medical task model; wherein the incremental task is split into a model capability evaluation subtask, a model routing scheduling subtask and a model incremental fusion subtask; the model capability evaluation subtask is used to evaluate the ability of the task model to perform different tasks to determine the target fusion task model; the model routing scheduling subtask is used to determine the fusion order of the target fusion task model; the model incremental fusion subtask is used to incrementally fuse the target fusion task model with the initial medical task model.

5. The method according to claim 4, characterized in that The agent that performs the model capability assessment subtask performs the model capability assessment subtask as follows: Based on the incremental tasks of the current medical center, multiple task models from other medical centers corresponding to the incremental tasks are selected from the model federation network as the task models to be fused; Each task model to be fused is scored according to a preset scoring method, and a task model to be fused with a score greater than or equal to a preset score is selected from the scoring results as the target fusion task model for incremental fusion; wherein the preset scoring method is: Among them, K H Represents the kernel matrix, which is used to characterize the sensitivity of the task model to data distribution; N A represents the number of neurons in the task model; d H (c N ,c1) represents the Hamming distance of the overlap between the activation of the Nth neuron and the first neuron node in the task model, c N represents the activation of the Nth neuron, and c1 represents the activation of the first neuron.

6. The method according to claim 5, characterized in that The agent that performs the model routing scheduling subtask performs the model routing scheduling subtask as follows: The fusion order of the target fusion task model during incremental fusion is determined according to a preset scheduling method, wherein the preset scheduling method is: Among them, C total represents the total cost of incremental fusion according to the fusion order, x ij represents the decision variable, x ij =1 means that the i-th task model is assigned the corresponding fusion scheme for incremental fusion with the j-th task model; c ij Represents the execution decision variable x ij The cost consumed by the corresponding fusion solution.

7. The method according to claim 6, characterized in that The agent that performs the model incremental fusion subtask performs the model incremental fusion subtask as follows: The neuron scoring of each target fusion task model is performed according to the preset neuron scoring method to obtain the score of each neuron in each target fusion model, wherein the preset neuron scoring method is: Among them, APE i,j represents the activation probability entropy of the jth neuron in the i-th neural network layer of the task model, l represents the total number of task categories, represents the probability that the jth neuron in the i-th neural network layer is activated under the k-th task; The neurons in each target fusion task model that are greater than or equal to the preset neuron score are taken as incremental neurons, and the parameters of the incremental neurons in each target fusion task model are weightedly fused with the corresponding neuron parameters in the initial medical task model in the fusion order to obtain the fused medical task model.

8. A medical data classification method, characterized in that: The method comprises: Step S1, obtaining medical data to be processed; Step S2: Classify the medical data to be processed using the medical task model obtained by the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 8.

10. An electronic device, characterized in that: include: one or more processors; as well as A memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1-8 by executing the executable instructions.

Images