Diagnosis prediction device and system based on knowledge caching and personalized federated learning

Abstract

The invention provides a diagnosis prediction device and system based on knowledge caching and personalized federated learning, and relates to the field of model learning and medical image diagnosis. The method comprises the following steps: receiving medical sample index information uploaded by a plurality of clients, and constructing a dynamic knowledge cache; creating an index relation mapping table from labels to indexes and from the indexes to hash values according to the medical sample index information; initializing each medical sample and establishing index relation mapping between each medical sample and R most similar samples; receiving a category probability vector and an index thereof uploaded by each client, and updating the knowledge cache; updating the index relation mapping table, and retrieving knowledge vectors of R most similar samples; generating comprehensive knowledge representation by adopting a weighted average method; distributing the comprehensive knowledge representation to each client for personalized federated learning; each trained client model is used for medical diagnosis prediction. According to the method, the problems of high communication cost, poor model structure compatibility and difficulty in acquiring the public data set are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of model learning and medical imaging diagnosis, and in particular to a diagnosis prediction device and system based on knowledge caching and personalized federated learning. Background Art

[0002] In recent years, with the widespread application of medical imaging technologies (such as X-ray, CT, MRI), the potential of artificial intelligence in medical imaging diagnosis has become increasingly apparent. However, the sensitivity of medical data and the strict requirements of relevant privacy regulations have limited the use of traditional centralized machine learning methods. Federated learning has become an effective means of data privacy protection by collaboratively training models on distributed devices and avoiding the sharing of original data.

[0003] In the prior art, the application of personalized federated learning in medical imaging diagnosis mainly adopts an architecture based on parameter interaction, which is: each client (such as a hospital imaging workstation) uses a unified model structure to train local medical imaging data, generates large-scale model parameters and uploads them to the central server; the server aggregates these parameters by weighted average, updates the global model and distributes it. To alleviate the lack of personalization, some methods introduce additional public data sets (such as public image libraries) to assist in training, or fine-tune the global model on the client to adapt to local needs. However, the architecture based on parameter interaction requires frequent transmission of large-scale model parameters, resulting in high communication costs between distributed medical devices; and the model structure has poor compatibility, and usually requires a unified model structure, which is difficult to adapt to the diverse characteristics of hardware performance and diagnostic needs of different medical institutions. In addition, some methods use additional public data sets, and obtaining such data in the medical field is difficult due to privacy restrictions.

[0004] In view of this, the applicant filed this application after studying the existing technology. Summary of the invention

[0005] The present invention aims to provide a diagnostic prediction device and system based on knowledge caching and personalized federated learning to solve the problems of high communication cost, poor model structure compatibility and difficulty in obtaining public data sets in the traditional federated learning architecture based on parameter interaction. The present invention introduces dynamic knowledge caching and multimodal feature fusion mechanism, combined with index relationship mapping and weighted knowledge fusion strategy, to achieve more efficient distributed medical imaging diagnostic model training, while meeting the strict requirements of privacy protection in the medical field.

[0006] In order to solve the above technical problems, the present invention is implemented through the following technical solutions: A diagnosis prediction device based on knowledge caching and personalized federated learning, applied to an edge server of a distributed medical network, comprising a dynamic knowledge cache construction unit, an index relationship mapping unit, a most similar sample mapping unit, a knowledge cache update unit, a knowledge vector retrieval unit, a comprehensive knowledge representation generation unit, and a personalized learning unit; The dynamic knowledge cache construction unit is used to receive medical sample index information uploaded by multiple clients and construct a dynamic knowledge cache; wherein the medical sample index information includes a hash value, index and label of the medical sample; The index relationship mapping unit is used to create an index relationship mapping table of label to index and index to hash value according to the medical sample index information, so as to associate each label with a corresponding index and each index with a corresponding hash value; The most similar sample mapping unit is used to initialize each medical sample and establish an index relationship mapping between each medical sample and R most similar samples; The knowledge cache updating unit is used to receive the category probability vector and its index uploaded by each client, so as to update the latest reasoning prediction result of the client to the knowledge cache; The knowledge vector retrieval unit is used to update the index relationship mapping table according to the updated knowledge cache, and retrieve the knowledge vectors of the R most similar samples of each medical sample; The comprehensive knowledge representation generation unit is used to generate a comprehensive knowledge representation by fusing the knowledge vectors using a weighted average method; The personalized learning unit is used to distribute the comprehensive knowledge representation to each client; wherein the comprehensive knowledge representation is used to trigger the client model of each client to perform personalized federated learning training of the distributed medical network based on the comprehensive knowledge representation, and perform medical diagnosis prediction based on the trained client model to output a prediction label.

[0007] Preferably, the medical sample index information is obtained by each client performing feature fusion and privacy encoding processing on the locally stored medical image samples, specifically: Each client reads a medical sample dataset in a local database; wherein the medical sample dataset includes medical image samples, patient metadata, and diagnosis labels, expressed as: ; in, represents a medical sample dataset; k∈{1,2,…,K} represents the kth client, and K represents the total number of clients in the distributed medical network; is the i-th image sample; for The corresponding patient metadata; for The corresponding diagnostic label; is the sample size; Each client uses its own pre-trained client model to process medical image samples and patient metadata to extract corresponding image features and text features; wherein the client model includes a convolutional neural network and a Deepseek model; the input medical image samples are processed by the convolutional neural network, and the spatial features of the image are extracted through convolution and pooling operations to capture the texture and structural information in the image, and the image features are obtained, which are expressed as: ; in, represents the image features of the i-th image sample of the k-th client; , are the pre-trained weight and bias parameters respectively; The patient metadata input is processed by the Deepseek model, and the text semantic features are extracted through matrix operations to reflect the individual characteristics of the patient. The text features are obtained, and the expression is: ; in, Represents patient metadata Text features of , are the pre-trained weight and bias parameters respectively; The image features and the text features are combined to obtain a comprehensive feature vector, which is expressed as: ; in, is the comprehensive feature vector corresponding to the i-th image sample; Represents a splicing operation; The differential privacy hash algorithm is used to encode and protect the comprehensive feature vector, that is, the comprehensive feature vector is mapped to a low-dimensional space through a random projection matrix, and then Gaussian noise perturbation is added to generate a hash value of a fixed length, which is expressed as: ; ; in, is the hash value corresponding to the i-th image sample of the k-th client; DPH represents the differential privacy hash algorithm; H is the random projection matrix; Indicates that the mean square is 0 and the variance is Gaussian noise; and is the differential privacy parameter; ΔF is the sensitivity of the feature, indicating the maximum range of variation of the feature; sign represents the encoding operation; Each client uploads the corresponding hash value, diagnostic label and index obtained from the comprehensive feature vector to the edge server through an encrypted channel.

[0008] Preferably, when initializing each medical sample, each medical sample is initialized as a knowledge vector of an all-zero vector, representing its initial state before training; the dimension of the knowledge vector is consistent with the number of categories of the diagnostic label.

[0009] Preferably, when establishing an index relationship mapping between each medical sample and the R most similar samples, The cosine similarity between the hash values ​​of all medical samples is calculated, and the time decay factor is introduced to reduce the weight of the medical sample data uploaded earlier, the R most similar samples are screened out, and the improved HNSW algorithm is used to accelerate the retrieval process. The expression is: ; in, represents the R most similar samples of the i-th image sample of the k-th client; Represents two hash values , cosine similarity of; k, represents the index of the client; i and j represent the index of the image sample; It represents the time decay rate, which is used to control the influence of historical data; t represents the current timestamp; Indicates The upload timestamp of the jth image sample of a client; Top-R means selecting the top R samples with the highest similarity.

[0010] Preferably, the category probability vector is the predicted category probability corresponding to the medical sample generated by each client using the client model to infer and predict the input medical sample through forward propagation; the number of elements in the category probability vector is consistent with the number of categories.

[0011] Preferably, the expression of the comprehensive knowledge representation is: ; ; in, representing the comprehensive knowledge representation; The knowledge vector representing the i-th image sample of the k-th client, that is, the knowledge vector corresponding to the index (k, i); Represents the knowledge vector of the s-th similar sample; represents the weight of the corresponding knowledge vector; R represents the number of similar samples.

[0012] Preferably, the weight of the comprehensive knowledge representation The cosine similarity of the hash values ​​corresponding to the image samples and the similarity of the patient metadata are normalized and expressed as: ; ; in, represents the hash value of the i-th image sample of the k-th client; , Respectively represent the sth, The hash value of similar samples; cos represents cosine similarity; represents the patient metadata of the i-th image sample of the k-th client; , Respectively represent the sth, patient metadata for similar samples; Indicates similarity; represents the L2 norm; , Indicates the patient age of the corresponding sample, through patient metadata get; , The patient's condition severity vector for the corresponding sample is obtained by combining the patient metadata with the corresponding diagnostic label and scoring by experts.

[0013] Preferably, when each client performs personalized federated learning training of a distributed medical network, a comprehensive objective function is used to optimize the model; wherein the comprehensive objective function includes a supervision loss function and a distillation loss function, and is weighted and calculated by dynamic weights; The supervised loss function is calculated using cross entropy loss, and the model parameters are adjusted by comparing the difference between the model predicted category probability and the true label to ensure that the model prediction is consistent with the local diagnostic label; The distillation loss function uses KL divergence to measure the distribution difference between the local category probability vector and the comprehensive knowledge representation, so that the local reasoning result is consistent with the knowledge cache; the expressions are: ; ; ; in, represents a medical sample dataset; Represents the client model comprehensive objective function The trainable parameters of is the i-th image sample; is the true value of the label; Represents the true value of the c-th class label; represents the supervised loss function; express The predicted probability of Represents the softmax normalization function; represents the dynamic weight of the comprehensive objective function; KL represents the distillation loss function; represents the predicted probability on the knowledge cache; C represents the total number of label categories; Represents the predicted probability of the c-th class label; represents the predicted probability of the c-th class label on the knowledge cache; represents the probability prediction function.

[0014] Preferably, the dynamic weight of the comprehensive objective function Adjusted according to the heterogeneity of local medical sample data and the severity of the patient's condition, the formula is: ; in, Represents a local medical sample dataset Global data distribution on knowledge cache The KL divergence of Represents patient metadata The severity score of the disease is obtained by combining the patient metadata with the corresponding tags and scoring by experts; , are the weights corresponding to the KL divergence and the disease severity score; Each client calculates the gradient of the comprehensive objective function and updates the trainable parameters of the model according to the learning rate , the expression is: ; in, is the learning rate; ) is the comprehensive objective function right gradient.

[0015] The present invention also provides a diagnosis prediction system based on knowledge caching and personalized federated learning, comprising an edge server and a plurality of clients; each client is connected to the edge server via communication; The client is used to read the medical sample data set in the local database, and use the pre-trained client model to process the medical sample data set to extract the corresponding features for splicing and fusion to obtain a comprehensive feature vector; then the differential privacy hash algorithm is used to encode and protect the comprehensive feature vector to obtain medical sample index information, which is uploaded to the edge server; wherein the medical sample index information includes the hash value, index and label of the medical sample; The edge server is used to build a dynamic knowledge cache according to the medical sample index information uploaded by multiple clients; and create an index relationship mapping table from label to index and from index to hash value according to the medical sample index information, so as to associate each label with a corresponding index and each index with a corresponding hash value; then, initialize each medical sample and establish an index relationship mapping between each medical sample and R most similar samples; The client is used to use the client model to infer the input medical sample data and generate a category probability vector through forward propagation, and upload it to the edge server together with the corresponding index; The edge server is used to receive the category probability vectors and their indexes uploaded by multiple clients, so as to update the latest inference prediction results of the client to the knowledge cache; update the index relationship mapping table according to the updated knowledge cache, and retrieve the knowledge vectors of the R most similar samples of each medical sample; then use the weighted average method to fuse the knowledge vectors to generate a comprehensive knowledge representation, and distribute the comprehensive knowledge representation to each client; The client is used to receive the comprehensive knowledge representation distributed by the edge server and perform personalized federated learning and optimization training of the distributed medical network; the trained model is used for remote medical imaging diagnosis prediction.

[0016] In summary, compared with the prior art, the present invention has the following beneficial effects: The edge server of the present invention weightedly fuses the knowledge of the R most similar samples based on the similarity of hash values ​​and metadata (such as age and severity of the disease) to form personalized guidance for each client, which is particularly suitable for disease screening and remote diagnosis in distributed medical networks, and provides an efficient and secure solution for smart medical care.

[0017] The present invention reduces the communication overhead between distributed medical devices, supports heterogeneous model architecture, adapts to the computing capabilities of different devices, realizes personalized medical imaging diagnosis without the need for public data sets, and improves the adaptability and prediction accuracy of client models for different patient groups.

[0018] The present invention encodes the comprehensive features of medical images and patient metadata into fixed-length hash values ​​through the differential privacy hash algorithm (DPH) and adds Gaussian noise perturbation to effectively prevent the leakage of original data. Compared with the traditional method of directly transmitting complete model parameters, it greatly improves privacy security.

[0019] The client of the present invention only uploads category probability vectors of the same size as the number of categories (e.g., only two values ​​for two categories of problems), and performs global fusion with the knowledge cache and time decay factor of the edge server, reducing the communication cost to less than one thousandth of that of traditional federated learning. It also supports heterogeneous model deployment and adapts to different hardware requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A schematic diagram of the structure of a diagnostic prediction device based on knowledge caching and personalized federated learning provided in Example 1.

[0022] Figure 2 A flowchart of a diagnostic prediction device based on knowledge caching and personalized federated learning provided in Example 1.

[0023] Figure 3 A schematic diagram of a diagnosis prediction system based on knowledge caching and personalized federated learning provided in Example 2.

[0024] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0026] Embodiment 1 Embodiment 1 of the present invention provides a diagnosis prediction device based on knowledge caching and personalized federated learning, which can be implemented by a diagnosis prediction device based on knowledge caching and personalized federated learning (hereinafter referred to as the diagnosis prediction device), and in particular, executed by one or more processors in the diagnosis prediction device.

[0027] In this embodiment, the diagnostic prediction device may be an electronic device equipped with a processor, the processor having a computer program of the diagnostic prediction device based on knowledge caching and personalized federated learning and the computer program can be executed, such as a computer, a smart phone, a smart tablet, a workstation, etc., which is not limited here.

[0028] In this embodiment, knowledge caching is an important feature of the distributed medical network system, which allows the edge server to store and update the medical image features and knowledge learned from the client (such as portable ultrasound machine, hospital imaging workstation). Through this mechanism, the system can quickly call and infer new medical image data, improving processing speed and accuracy. At the same time, knowledge caching also helps to reduce the computing pressure of the edge server and improve the overall performance and efficiency of the system.

[0029] In this embodiment, personalized federated learning is another core feature of the distributed medical network system, which allows the client to perform a certain degree of model training and learning locally. In this way, the system can make full use of the specific data and characteristics of each client to improve the accuracy and adaptability of the model. Federated learning also ensures the privacy and security of the data, because the original data will not leave the client, and only the processed features or model parameters will be transmitted to the edge server for further analysis and training.

[0030] like Figure 1-Figure 2 As shown, a diagnosis and prediction device based on knowledge caching and personalized federated learning is applied to the edge server of a distributed medical network, including a dynamic knowledge cache construction unit, an index relationship mapping unit, a most similar sample mapping unit, a knowledge cache update unit, a knowledge vector retrieval unit, a comprehensive knowledge representation generation unit and a personalized learning unit.

[0031] Dynamic knowledge cache construction unit: used to receive medical sample index information uploaded by multiple clients and construct a dynamic knowledge cache; wherein the medical sample index information includes the hash value, index and label of the medical sample.

[0032] In this embodiment, the overall architecture of the distributed medical network system includes multiple clients and an edge server, and each client is connected to the edge server through communication. These components achieve efficient and personalized model training through a distributed computing framework and privacy protection mechanism, solving the problems of high communication costs, insufficient model heterogeneity, and limited personalization capabilities of traditional methods.

[0033] When the system starts, each client extracts features from the locally stored medical image data and performs privacy protection processing to establish a foundation for subsequent knowledge sharing. In this embodiment, the medical sample index information is obtained by each client performing feature fusion and privacy encoding processing on the locally stored medical image samples, specifically: Each client reads a medical sample dataset in a local database; wherein the medical sample dataset includes medical image samples (such as CT scan images), patient metadata (such as age, gender, medical history), and diagnostic labels (such as "positive lung nodules" or "normal"), expressed as: ; in, represents a medical sample dataset; k∈{1,2,…,K} represents the kth client, and K represents the total number of clients in the distributed medical network; is the i-th image sample; for The corresponding patient metadata; for The corresponding diagnostic label; is the sample size; Each client uses its own pre-trained client model to process medical image samples and patient metadata to extract corresponding image features and text features; wherein the client model includes a convolutional neural network and a Deepseek model. For image samples, the convolutional neural network processes the input medical image samples, extracts the spatial features of the image through convolution and pooling operations to capture the texture and structural information in the image, and obtains the image features, which are expressed as: ; in, represents the image features of the i-th image sample of the k-th client; , are the pre-trained weight and bias parameters respectively; For metadata, the patient metadata is processed by the Deepseek model (such as Deepseek R1), and the text semantic features are extracted through matrix operations to reflect the individual characteristics of the patient. The text features are obtained, and the expression is: ; in, Represents patient metadata Text features of , are the pre-trained weight and bias parameters respectively; After the extraction is completed, the image features and the text features are spliced ​​and fused to obtain a comprehensive feature vector so as to uniformly process the two types of information. The expression is: ; in, is the comprehensive feature vector corresponding to the i-th image sample; Represents a concatenation operation.

[0034] Due to the sensitivity of medical data, directly uploading the original features may lead to privacy leakage, so the client uses a differential privacy hash algorithm to encode and protect the comprehensive feature vector. That is, the comprehensive feature vector is mapped to a low-dimensional space through a random projection matrix, and then Gaussian noise perturbation is added to generate a fixed-length hash value, which is expressed as: ; ; in, is the hash value corresponding to the i-th image sample of the k-th client; DPH represents the differential privacy hashing algorithm; H is a random projection matrix (e.g., it can be generated by the Johnson-Lindenstrauss theorem); Indicates that the mean square is 0 and the variance is Gaussian noise; and is the differential privacy parameter; ΔF is the sensitivity of the feature, which indicates the maximum range of variation of the feature (for example, it can be taken as the maximum L2 norm difference of the feature vector of the training set); sign indicates the encoding operation.

[0035] In this embodiment, the Differential Privacy Hashing (DPH) algorithm is an innovative method that combines differential privacy protection technology with the hash algorithm, aiming to achieve data de-identification while strictly protecting privacy security. The principle is to add random noise (such as Laplace or Gaussian noise) that satisfies differential privacy in the traditional hashing process, so that the output result cannot be reversed to the original data, ensuring that the hash output distribution of adjacent data sets (only one record different) is almost the same.

[0036] This method ensures data privacy while preserving the similarity relationship between samples. After encoding, the client will hash the value , sample index (k,i) and diagnostic label Upload to the edge server through an encrypted channel (such as TLS protocol) to provide secure data for subsequent cache construction.

[0037] Index relationship mapping unit: used to create an index relationship mapping table from label to index and from index to hash value according to the medical sample index information, so as to associate each label with the corresponding index and each index with the corresponding hash value.

[0038] In this embodiment, after receiving the hash value, index and label uploaded by the client, the edge server builds a dynamic knowledge cache (KC) to efficiently organize distributed data and support knowledge sharing in subsequent training.

[0039] First, the server creates a mapping table from labels to indices, associating each diagnostic label with the corresponding sample index. For example, the label "positive lung nodules" is mapped to the index of all related samples to facilitate fast retrieval of samples by category. The update method is as follows: ; in, Diagnostic label The corresponding index set; It is a set merge operation; is the sample index, indicating the i-th image sample of the k-th client.

[0040] Next, the hash value of each sample is stored in the index map, and a lookup table from the sample index to the hash value is established to provide a data basis for subsequent similarity calculations. The storage method is as follows: ; in, Index for samples The corresponding hash value is , A collection of hash values.

[0041] The most similar sample mapping unit is used to initialize each medical sample and establish an index relationship mapping between each medical sample and the R most similar samples.

[0042] In this unit, the server initializes each medical sample as a knowledge vector of an all-zero vector, representing its initial state before training; the dimension of the knowledge vector is consistent with the number of categories of the diagnostic label, and the initialization process is as follows: ; in, Index for samples The corresponding knowledge vector, C is the total number of diagnostic categories, such as "positive" and "negative" corresponding to C = 2, Represents a real number.

[0043] Next, an index relationship mapping between each medical sample and the R most similar samples is established to support knowledge fusion.

[0044] The cosine similarity between the hash values ​​of all medical samples is calculated, and a time decay factor is introduced to reduce the weight of the medical sample data uploaded earlier (for example, the weight of samples uploaded more than one month is halved), the R most similar samples are screened out, and the improved HNSW algorithm is used to accelerate the retrieval process. The expression is: ; in, represents the R most similar samples of the i-th image sample of the k-th client; Represents two hash values , cosine similarity of; k, represents the index of the client; i and j represent the index of the image sample; It represents the time decay rate, which is used to control the influence of historical data; t represents the current timestamp; Indicates The upload timestamp of the jth image sample of a client; Indicates selecting the first R samples with the highest similarity.

[0045] In this embodiment, the improved HNSW (Hierarchical Navigable Small World) algorithm may adopt a graph traversal strategy improvement method to perform directional heuristic pruning; or may adopt an index structure enhancement method of dynamic hierarchical weight adjustment and incremental quantization coding.

[0046] Through the above index mapping, knowledge cache provides efficient data support for distributed training.

[0047] Knowledge cache update unit: used to receive the category probability vector and its index uploaded by each client to update the client's latest inference prediction result to the knowledge cache.

[0048] This step is the model training phase of each client to extract local knowledge. At the beginning of each round of model training, each client uses the client model to reason about image samples and extract knowledge representation to participate in global knowledge fusion. The client selects the appropriate model architecture based on its own hardware performance, and these models do not need to be consistent with other clients.

[0049] The client model is used to infer the input medical samples and generate the predicted category probability corresponding to the medical samples through forward propagation, that is, the category probability vector logits. The number of elements in the category probability vector is consistent with the number of categories. For example, for a 2-category problem, logits is 2 values. The data volume of the category probability vector is much smaller than the complete model parameters. The client uploads it to the server together with the index (k, i), which greatly reduces the communication overhead. The category probability vector is expressed as: ; in, is the logits vector of the i-th sample of the k-th client; Trainable parameters representing the comprehensive objective function of the client model; is the i-th image sample; Represents the probability prediction function of the client model.

[0050] After receiving the logits uploaded by each client, the server first updates the knowledge cache and records the latest inference results in the cache. The update method is as follows: .

[0051] Knowledge vector retrieval unit: used to update the index relationship mapping table according to the updated knowledge cache, and retrieve the knowledge vectors of the R most similar samples for each medical sample.

[0052] In this unit, the knowledge set of the R most similar samples of each sample is retrieved according to the index relationship mapping. For example, the logits of the R=10 samples most similar to the current sample are extracted from the cache. The results are as follows: ; in, Represents the knowledge vector of the Rth similar sample.

[0053] Comprehensive knowledge representation generation unit: used to generate comprehensive knowledge representation by fusing the knowledge vectors using a weighted average method.

[0054] Furthermore, the expression of the comprehensive knowledge representation is: ; , that is, the weights need to meet the normalization conditions.

[0055] in, representing the comprehensive knowledge representation; The knowledge vector representing the i-th image sample of the k-th client, that is, the knowledge vector corresponding to the index (k, i); Represents the knowledge vector of the s-th similar sample; represents the weight of the corresponding knowledge vector; R represents the number of similar samples.

[0056] The weight of the comprehensive knowledge representation The cosine similarity of the hash values ​​corresponding to the image samples and the patient metadata similarity are normalized to reflect the semantic relevance between samples. The patient metadata similarity is calculated based on the age difference and disease severity of the patients. For example, samples with an age difference of less than 5 years have a higher similarity, and samples with similar disease scores have a higher weight.

[0057] The expression is: ; ; in, represents the hash value of the i-th image sample of the k-th client; , Respectively represent the sth, The hash value of similar samples; cos represents cosine similarity; represents the patient metadata of the i-th image sample of the k-th client; , Respectively represent the sth, patient metadata for similar samples; Indicates similarity; represents the L2 norm; , Indicates the patient age of the corresponding sample, through patient metadata get; , The patient's condition severity vector for the corresponding sample is obtained by combining the patient metadata with the corresponding diagnostic label and scoring by experts.

[0058] When experts grade, different diseases have corresponding diagnostic labels, and the expert team will assign severity scores to these labels. For example: Chronic diseases (such as hypertension): may be scored on a graded basis (grade 1 / grade 2 / grade 3).

[0059] Acute illness (such as myocardial infarction): combined with Killip classification or TIMI scoring system.

[0060] Complex diseases: quantify the risk of multiple organ failure through scoring tools such as APACHE II and SOFA.

[0061] Clinical experts (such as attending physicians and above) can also be invited to perform double-blind scoring of cases. Each expert independently quantifies the [symptom duration] and [number of organs affected] indicators in the metadata on a 0-10 scale based on the "ICD-11 Clinical Severity Assessment Guidelines", and finally takes a weighted average (weight: such as 0.6 for chief physicians / 0.4 for deputy chief physicians).

[0062] Personalized learning unit: used to distribute the comprehensive knowledge representation to each client; wherein the comprehensive knowledge representation is used to trigger the client model of each client to perform personalized federated learning training of the distributed medical network based on the comprehensive knowledge representation, and perform medical diagnosis prediction based on the trained client model to output the prediction label.

[0063] Comprehensive knowledge representation after fusion Distributed to the corresponding clients as guidance information for personalized training.

[0064] The client receives the comprehensive knowledge representation After that, the model is optimized in combination with local data. When training the client model, a comprehensive objective function is used for optimization. The comprehensive objective function includes a supervision loss function and a distillation loss function, and is calculated by dynamic weights.

[0065] The supervised loss function is calculated using cross entropy loss, and the model parameters are adjusted by comparing the difference between the model predicted category probability and the true label to ensure that the model prediction is consistent with the local diagnostic label; The distillation loss function uses KL divergence to measure the distribution difference between the local category probability vector and the comprehensive knowledge representation, so that the local reasoning result is consistent with the knowledge cache; the expressions are: ; ; ; in, represents a medical sample dataset; Represents the client model comprehensive objective function The trainable parameters of is the i-th image sample; is the true value of the label; Represents the true value of the c-th class label; represents the supervised loss function; express The predicted probability of Represents the softmax normalization function; represents the dynamic weight of the comprehensive objective function; KL represents the distillation loss function; represents the predicted probability on the knowledge cache; C represents the total number of label categories; Represents the predicted probability of the c-th class label; represents the predicted probability of the c-th class label on the knowledge cache; represents the probability prediction function.

[0066] To achieve personalization, the dynamic weights are adjusted according to the heterogeneity of local data and the severity of the patient's condition. For example, samples with large data distribution differences or severe conditions are given higher weights. That is, the dynamic weights of the comprehensive objective function are Adjusted according to the heterogeneity of local medical sample data and the severity of the patient's condition, the formula is: ; in, Represents a local medical sample dataset Global data distribution on knowledge cache The KL divergence of Represents patient metadata The severity score of the disease is obtained by combining the patient metadata with the corresponding tags and scoring by experts; , are the weights corresponding to the KL divergence and the disease severity score; Each client updates the model parameters using the gradient descent method by calculating the gradient of the comprehensive objective function and updating the trainable parameters of the model according to the learning rate. , the expression is: ; in, is the learning rate; ) is the comprehensive objective function right gradient.

[0067] After the above steps, the trained client models are used for remote medical diagnosis prediction and disease screening to improve the adaptability and prediction accuracy of the model for different patient groups. For example, on client 1, the patient's medical imaging data (such as CT images) is input into the model trained based on knowledge caching and personalized federated learning for diagnosis prediction, and the prediction results are obtained, such as "positive lung nodules" or "normal". The prediction results combine the knowledge uploaded and updated by each client, which improves the accuracy of the prediction results.

[0068] This technical solution achieves efficient distributed training while protecting privacy, and improves the personalized performance of the model through knowledge fusion.

[0069] In summary, compared with the prior art, the present invention has the following beneficial effects: In actual application scenarios, the device of the present invention can be deployed in a distributed hospital network and is suitable for disease screening and remote diagnosis. For example, in the task of lung nodule screening, portable ultrasound machines or imaging workstations in different hospitals are used as client devices, and patient metadata and diagnostic labels are input through text boxes, and personalized federated learning is completed through the above steps. The edge server realizes efficient knowledge sharing and fusion through the dynamic knowledge cache construction unit and the knowledge cache update unit, significantly reducing communication costs and improving model personalization performance.

[0070] In the dynamic knowledge cache construction unit, the comprehensive features of medical images and patient metadata are encoded into fixed-length hash values ​​through the differential privacy hash algorithm (DPH), and Gaussian noise perturbation is added to prevent the leakage of original data; the client only uploads the logits vector of the size of the number of categories, and combines the knowledge cache and time decay factor of the edge server for global fusion, and the communication cost is reduced to less than one thousandth of traditional federated learning. At the same time, it supports heterogeneous model architecture and adapts to different hardware requirements. The server weights the knowledge of the R most similar samples based on the similarity of hash values ​​and metadata to form personalized guidance, providing efficient and secure solutions for smart medical care.

[0071] The present invention adopts an incremental knowledge update mechanism to initialize the knowledge vector of each medical sample as an all-zero vector when it first enters the knowledge cache, then receives the latest reasoning results uploaded by the client, updates the knowledge vector of the corresponding medical sample, and timestamps the time point of each update to ensure that the data in the knowledge cache always reflects the latest reasoning results.

[0072] The present invention adopts an adaptive similarity adjustment system, generates an initial similarity matrix by calculating the cosine similarity between hash values ​​of medical samples, generates an adjusted similarity matrix by introducing a time decay factor, selects the top R samples with the highest similarity from the adjusted similarity matrix, and dynamically adjusts the decay rate of the time decay factor according to the difference between the upload timestamp and the current timestamp to ensure that the influence of historical data gradually weakens over time.

[0073] The present invention adopts a multi-dimensional weight distribution mechanism, generates similarity weights by calculating the cosine similarity between hash values ​​of image samples, generates metadata weights by calculating the similarity of patient age and disease severity vectors, and obtains the final weight by normalizing the similarity weights and metadata weights, combining image features and individual patient characteristics to improve the accuracy of knowledge vector fusion. The comprehensive knowledge representation is distributed to all clients for global model updates, and is trimmed or enhanced according to the hardware performance and diagnostic requirements of the client and distributed to specific clients for personalized model optimization. The hardware performance and diagnostic requirements of different medical institutions are met through differentiated distribution strategies.

[0074] The present invention adopts a gradient dynamic adjustment mechanism, generates a gradient update direction by calculating the gradient of the comprehensive objective function for the model's trainable parameters, dynamically adjusts the learning rate to generate a gradient update step size according to the heterogeneity of local data and the patient's disease severity score, and improves the efficiency and effect of model optimization by combining local data characteristics and individual patient characteristics.

[0075] Embodiment 2 like Figure 3 As shown, the second embodiment of the present invention also provides a diagnosis prediction system based on knowledge caching and personalized federated learning, including an edge server and multiple clients; each client is connected to the edge server through communication; wherein the client is used to read the medical sample data set in the local database, and use the pre-trained client model to process the medical sample data set to extract the corresponding features for splicing and fusion to obtain a comprehensive feature vector; then the differential privacy hash algorithm is used to encode and protect the comprehensive feature vector to obtain medical sample index information, which is uploaded to the edge server; wherein the medical sample index information includes the hash value, index and label of the medical sample; The edge server is used to build a dynamic knowledge cache according to the medical sample index information uploaded by multiple clients; and create an index relationship mapping table from label to index and from index to hash value according to the medical sample index information, so as to associate each label with a corresponding index and each index with a corresponding hash value; then, initialize each medical sample and establish an index relationship mapping between each medical sample and R most similar samples; The client is used to use the client model to infer the input medical sample data and generate a category probability vector through forward propagation, and upload it to the edge server together with the corresponding index; The edge server is used to receive the category probability vectors and their indexes uploaded by multiple clients, so as to update the latest inference prediction results of the client to the knowledge cache; update the index relationship mapping table according to the updated knowledge cache, and retrieve the knowledge vectors of the R most similar samples of each medical sample; then use the weighted average method to fuse the knowledge vectors to generate a comprehensive knowledge representation, and distribute the comprehensive knowledge representation to each client; The client is used to receive the comprehensive knowledge representation distributed by the edge server and perform personalized federated learning and optimization training of the distributed medical network; the trained model is used for remote medical imaging diagnosis prediction.

[0076] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0077] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0078] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0079] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0080] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0081] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0082] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A diagnostic prediction device based on knowledge caching and personalized federated learning, applied to the edge server of a distributed medical network, characterized in that: It includes a dynamic knowledge cache construction unit, an index relationship mapping unit, a most similar sample mapping unit, a knowledge cache update unit, a knowledge vector retrieval unit, a comprehensive knowledge representation generation unit, and a personalized learning unit; The dynamic knowledge cache construction unit is used to receive medical sample index information uploaded by multiple clients and construct a dynamic knowledge cache; wherein the medical sample index information includes a hash value, index and label of the medical sample; The index relationship mapping unit is used to create an index relationship mapping table of label to index and index to hash value according to the medical sample index information, so as to associate each label with a corresponding index and each index with a corresponding hash value; The most similar sample mapping unit is used to initialize each medical sample and establish an index relationship mapping between each medical sample and R most similar samples; The knowledge cache updating unit is used to receive the category probability vector and its index uploaded by each client, so as to update the latest reasoning prediction result of the client to the knowledge cache; The knowledge vector retrieval unit is used to update the index relationship mapping table according to the updated knowledge cache, and retrieve the knowledge vectors of the R most similar samples of each medical sample; The comprehensive knowledge representation generation unit is used to generate a comprehensive knowledge representation by fusing the knowledge vectors using a weighted average method; The personalized learning unit is used to distribute the comprehensive knowledge representation to each client; wherein the comprehensive knowledge representation is used to trigger the client model of each client to perform personalized federated learning training of the distributed medical network based on the comprehensive knowledge representation, and perform medical diagnosis prediction based on the trained client model to output a prediction label.

2. A diagnostic prediction device based on knowledge caching and personalized federated learning according to claim 1, characterized in that ,The medical sample index information is obtained by each client performing feature fusion and privacy encoding processing on the ,locally stored medical image samples, specifically: Each client reads a medical sample dataset in a local database; wherein the medical sample dataset includes medical image samples, patient metadata, and diagnosis labels, expressed as: ； in, represents a medical sample dataset; k∈{1,2,…,K} represents the kth client, and K represents the total number of clients in the distributed medical network; is the i-th image sample; for The corresponding patient metadata; for The corresponding diagnostic label; is the sample size; Each client uses its own pre-trained client model to process medical image samples and patient metadata to extract corresponding image features and text features; wherein the client model includes a convolutional neural network and a Deepseek model; the input medical image samples are processed by the convolutional neural network, and the spatial features of the image are extracted through convolution and pooling operations to capture the texture and structural information in the image, and the image features are obtained, which are expressed as: ； in, represents the image features of the i-th image sample of the k-th client; , are the pre-trained weight and bias parameters respectively; The patient metadata input is processed by the Deepseek model, and the text semantic features are extracted through matrix operations to reflect the individual characteristics of the patient. The text features are obtained, and the expression is: ； in, Represents patient metadata Text features of , are the pre-trained weight and bias parameters respectively; The image features and the text features are combined to obtain a comprehensive feature vector, which is expressed as: ； in, is the comprehensive feature vector corresponding to the i-th image sample; Represents a splicing operation; The differential privacy hash algorithm is used to encode and protect the comprehensive feature vector, that is, the comprehensive feature vector is mapped to a low-dimensional space through a random projection matrix, and then Gaussian noise perturbation is added to generate a hash value of a fixed length, which is expressed as: ； ； in, is the hash value corresponding to the i-th image sample of the k-th client; DPH represents the differential privacy hash algorithm; H is the random projection matrix; Indicates that the mean square is 0 and the variance is Gaussian noise; and is the differential privacy parameter; ΔF is the sensitivity of the feature, indicating the maximum range of variation of the feature; sign represents the encoding operation; Each client uploads the corresponding hash value, diagnostic label and index obtained from the comprehensive feature vector to the edge server through an encrypted channel.

3. A diagnostic prediction device based on knowledge caching and personalized federated learning according to claim 1, characterized in that When initializing each medical sample, each medical sample is initialized as a knowledge vector of an all-zero vector, representing its initial state before training; the dimension of the knowledge vector is consistent with the number of categories of the diagnostic label.

4. A diagnostic prediction device based on knowledge caching and personalized federated learning according to claim 1, characterized in that ,When establishing the index relationship mapping between each medical sample and the R most similar samples, The cosine similarity between the hash values ​​of all medical samples is calculated, and the time decay factor is introduced to reduce the weight of the medical sample data uploaded earlier, the R most similar samples are screened out, and the improved HNSW algorithm is used to accelerate the retrieval process. The expression is: ； in, represents the R most similar samples of the i-th image sample of the k-th client; Represents two hash values , cosine similarity of; k, represents the index of the client; i and j represent the index of the image sample; It represents the time decay rate, which is used to control the influence of historical data; t represents the current timestamp; Indicates The upload timestamp of the jth image sample of a client; Indicates selecting the first R samples with the highest similarity.

5. A diagnostic prediction device based on knowledge caching and personalized federated learning according to claim 1, characterized in that , the category probability vector is the predicted category probability corresponding to the medical sample generated by each client using the client model to infer and predict the input medical sample and through forward propagation; the number of elements in the category probability vector is consistent with the number of categories.

6. A diagnostic prediction device based on knowledge caching and personalized federated learning according to claim 1, characterized in that , the expression of the comprehensive knowledge representation is: ； ； in, representing the comprehensive knowledge representation; The knowledge vector representing the i-th image sample of the k-th client, that is, the knowledge vector corresponding to the index (k, i); Represents the knowledge vector of the s-th similar sample; represents the weight of the corresponding knowledge vector; R represents the number of similar samples.

7. A diagnostic prediction device based on knowledge caching and personalized federated learning according to claim 6, characterized in that , the weight of the comprehensive knowledge representation The cosine similarity of the hash values ​​corresponding to the image samples and the similarity of the patient metadata are normalized and expressed as: ； ； in, represents the hash value of the i-th image sample of the k-th client; , Respectively represent the sth, The hash value of similar samples; cos represents cosine similarity; represents the patient metadata of the i-th image sample of the k-th client; , Respectively represent the sth, patient metadata for similar samples; Indicates similarity; represents the L2 norm; , Indicates the patient age of the corresponding sample, through patient metadata get; , The patient's condition severity vector for the corresponding sample is obtained by combining the patient metadata with the corresponding diagnostic label and scoring by experts.

8. A diagnostic prediction device based on knowledge caching and personalized federated learning according to claim 1, characterized in that ,When each client performs personalized federated learning training of the distributed medical network, a comprehensive objective function is used to optimize the model; ,wherein, the comprehensive objective function includes a supervision loss function and a distillation loss function, and is calculated by dynamic weights; The supervised loss function is calculated using cross entropy loss, and the model parameters are adjusted by comparing the difference between the model predicted category probability and the true label to ensure that the model prediction is consistent with the local diagnostic label; The distillation loss function uses KL divergence to measure the distribution difference between the local category probability vector and the comprehensive knowledge representation, so that the local reasoning result is consistent with the knowledge cache; the expressions are: ； ； ； in, represents a medical sample dataset; Represents the client model comprehensive objective function The trainable parameters of is the i-th image sample; is the true value of the label; Represents the true value of the c-th class label; represents the supervised loss function; express The predicted probability of Represents the softmax normalization function; represents the dynamic weight of the comprehensive objective function; KL represents the distillation loss function; represents the predicted probability on the knowledge cache; C represents the total number of label categories; Represents the predicted probability of the c-th class label; represents the predicted probability of the c-th class label on the knowledge cache; represents the probability prediction function.

9. A diagnostic prediction device based on knowledge caching and personalized federated learning according to claim 8, characterized in that , the dynamic weight of the comprehensive objective function Adjusted according to the heterogeneity of local medical sample data and the severity of the patient's condition, the formula is: ； in, Represents a local medical sample dataset Global data distribution on knowledge cache KL divergence of Represents patient metadata The severity score of the disease is obtained by combining the patient metadata with the corresponding tags and scoring by experts; , are the weights corresponding to the KL divergence and the disease severity score; Each client calculates the gradient of the comprehensive objective function and updates the trainable parameters of the model according to the learning rate , the expression is: ； in, is the learning rate; ) is the comprehensive objective function right gradient.

10. A diagnosis prediction system based on knowledge caching and personalized federated learning, characterized in that: It includes an edge server and a plurality of clients; each client is connected to the edge server via communication; The client is used to read the medical sample data set in the local database, and use the pre-trained client model to process the medical sample data set to extract the corresponding features for splicing and fusion to obtain a comprehensive feature vector; then the differential privacy hash algorithm is used to encode and protect the comprehensive feature vector to obtain medical sample index information, which is uploaded to the edge server; wherein the medical sample index information includes the hash value, index and label of the medical sample; The edge server is used to build a dynamic knowledge cache according to the medical sample index information uploaded by multiple clients; and create an index relationship mapping table from label to index and from index to hash value according to the medical sample index information, so as to associate each label with a corresponding index and each index with a corresponding hash value; then, initialize each medical sample and establish an index relationship mapping between each medical sample and R most similar samples; The client is used to use the client model to infer the input medical sample data and generate a category probability vector through forward propagation, and upload it to the edge server together with the corresponding index; The edge server is used to receive the category probability vectors and their indexes uploaded by multiple clients, so as to update the latest inference prediction results of the client to the knowledge cache; update the index relationship mapping table according to the updated knowledge cache, and retrieve the knowledge vectors of the R most similar samples of each medical sample; then use the weighted average method to fuse the knowledge vectors to generate a comprehensive knowledge representation, and distribute the comprehensive knowledge representation to each client; The client is used to receive the comprehensive knowledge representation distributed by the edge server and perform personalized federated learning and optimization training of the distributed medical network; the trained model is used for remote medical imaging diagnosis prediction.

Images