Chronic disease model lightweight promotion method and system based on big data

Abstract

The invention relates to the technical field of artificial intelligence and medical big data, and discloses a chronic disease model lightweight promotion method and system.The chronic disease model lightweight promotion method based on big data comprises the steps that differential privacy protection multi-stage federal distillation is adopted; enabling the medical institutions to cooperatively train the basic model under the condition of not sharing original data; grouping medical institutions based on geographic positions and disease spectrum features, and constructing a regional adaptive layer; carrying out resource portraying on the terminal equipment and generating a model variant with corresponding complexity; monitoring an equipment resource state in real time, and dynamically selecting a proper model variant; continuous optimization is realized through a differential privacy protection model updating mechanism; according to the method, the privacy of the patient is protected, and meanwhile, light weight and wide popularization of the chronic disease prediction model are realized, so that the model can efficiently run on various terminals from a high-performance server to resource-limited edge equipment and the like.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and medical big data technology, and more specifically, to a method and system for lightweight promotion of chronic disease models based on big data. Background Art

[0002] As chronic disease management becomes increasingly important, big data-based chronic disease prediction models are playing an increasingly crucial role in supporting medical decision-making and health management. By analyzing a patient's historical health data, lifestyle habits, environmental factors, and other multi-dimensional information, these models can predict disease risks, development trends, and treatment outcomes, providing scientific decision-making support for both medical staff and patients.

[0003] However, the practical application of chronic disease prediction models still faces numerous challenges. The high sensitivity of medical data makes it difficult for medical institutions to directly share raw patient data, leading to data silos. Significant differences in disease spectrum, population characteristics, and treatment practices exist across regions, making it difficult for models to maintain good performance across them. Furthermore, the computing power of equipment varies significantly within and across medical institutions, raising the question of how to flexibly deploy models, from high-performance servers to resource-constrained edge devices.

[0004] To address these issues, new approaches to model lightweighting and generalization are needed, based on technologies such as federated learning, knowledge distillation, and differential privacy. These technologies include: federated learning allows multiple parties to collaboratively train models without sharing original data; knowledge distillation transfers knowledge from complex models to simpler ones; differential privacy protects data privacy by adding noise; and model lightweighting aims to reduce the computational complexity of models so they can run on resource-constrained devices. Summary of the Invention

[0005] The present invention provides a lightweight promotion method and system for chronic disease models based on big data, which solves the technical problems in related technologies such as medical data silos, heterogeneity between medical institutions, diversity of computing resources, difficulty in model updating and maintenance, and difficulty in balancing privacy protection and model performance.

[0006] The present invention provides a lightweight promotion method for chronic disease models based on big data, comprising the following steps:

[0007] Using multi-level federated distillation with differential privacy protection, medical institutions can collaborate on training basic models without sharing raw data.

[0008] Based on the basic model, medical institutions are grouped according to their geographical location and disease spectrum characteristics. A regional adaptive layer is constructed for each group to achieve regional adaptability of the model and obtain a regional adaptive model.

[0009] Based on the regional adaptability model, we perform resource profiling on terminal devices and classify them based on federated clustering. We generate model variants of corresponding complexity for different resource levels and ensure knowledge transfer between model variants through progressive knowledge distillation.

[0010] Based on the generated model variants, the device resource status is monitored in real time, and the model variant version that best suits the current resource status is selected based on the scoring function to achieve dynamic deployment;

[0011] Based on the deployed model variants, local feedback data from each medical institution is collected through a differentially privacy-protected model update mechanism to achieve continuous optimization of the model.

[0012] In a preferred embodiment, the step of adopting multi-level federated distillation with differential privacy protection includes:

[0013] Each medical institution uses local patient data to train the initial model;

[0014] Each medical institution uses a public test dataset to generate statistical information about the model output and adds noise that satisfies differential privacy.

[0015] The central server collects the differentially private output statistics generated by each medical institution and integrates this information using a weighted average method;

[0016] The central server uses the integrated global output statistics as the output of the teacher model to train a student model with fewer parameters and lower computational complexity;

[0017] The central server distributes the trained lightweight student model as the basic model to each participating institution.

[0018] In a preferred embodiment, the step of grouping medical institutions according to geographical location and disease spectrum characteristics includes:

[0019] All participating medical institutions were divided into multiple groups based on their geographical location, service population characteristics, disease spectrum distribution, and other information;

[0020] For each group, the medical institutions within it use the distributed lightweight basic model as the initial model, fine-tune it based on their respective local data, and then use the federated averaging algorithm to generate a group-specific model;

[0021] Knowledge exchange between groups is carried out through cross-regional knowledge distillation technology, so that the models of each group can learn from each other while maintaining their respective regional characteristics.

[0022] In a preferred embodiment, the step of constructing a regional adaptation layer for each group includes:

[0023] On the basis of the basic model, a lightweight regional adaptation layer is added to each group to form a complete regional adaptation model;

[0024] The regional adaptation layer adopts at least one of a residual adaptation structure, a feature conversion structure, an attention modulation structure or a conditional normalization structure;

[0025] The degree of regional specialization is controlled by the regional specialization coefficient, which is a fixed value or a dynamic parameter that is automatically adjusted according to the characteristics of the input data.

[0026] In a preferred embodiment, the step of obtaining the regional adaptability model includes:

[0027] The central server combines the basic model with the corresponding regional adaptive layer to generate a lightweight model for each institution that is suitable for its regional characteristics;

[0028] For medical groups with multiple branches, a comprehensive model with multiple regional adaptive layers is generated;

[0029] For institutions located at the boundary of regions, a hybrid model is generated that fuses specialized layers from multiple adjacent regions.

[0030] In a preferred embodiment, the step of performing resource profiling on the terminal device and classifying the terminal device based on federated clustering includes:

[0031] Collect resource information of various terminal devices in each medical institution, including processor performance, memory capacity, storage space, power status and network connection;

[0032] Integrate the resource information of all devices into a device resource profile;

[0033] The federated K-means clustering algorithm is used to classify device resource profiles and divide devices into three resource levels: high, medium, and low.

[0034] In a preferred embodiment, the step of generating model variants of corresponding complexity for different resource levels includes:

[0035] For high-resource-level devices, the regional adaptive model is directly used as a high-complexity model variant;

[0036] For medium-resource devices, network pruning and selective quantization techniques are used to generate medium-complexity model variants.

[0037] For low-resource devices, model distillation and structural reorganization techniques are used to generate low-complexity model variants;

[0038] Based on the hardware characteristics of the target device, each model variant is optimized for computational graphs, memory access, and dedicated instruction sets.

[0039] In a preferred embodiment, the step of selecting the model variant version that best suits the current resource status based on the scoring function includes:

[0040] Run a lightweight resource monitoring module on the device to collect device operating status information in real time;

[0041] Based on the resource profile of the current device, select the most suitable version from multiple model variants;

[0042] The scoring function comprehensively considers model performance and resource consumption, and balances accuracy and resource consumption through weight parameters;

[0043] When resource status changes significantly, a smooth switching strategy is adopted to achieve seamless transition between model variants.

[0044] In a preferred embodiment, the model update mechanism for differential privacy protection includes:

[0045] Each medical institution collects local prediction feedback data to form a feedback data set;

[0046] Use local feedback data to fine-tune the current model and generate an improved local model;

[0047] Extract the knowledge increments of the local model improvement and add noise that satisfies differential privacy;

[0048] The central server collects the knowledge increments submitted by each institution and aggregates them using a weighted average method to generate global model updates;

[0049] The central server converts the global knowledge increment into model parameter updates and sends them to each organization.

[0050] In a preferred embodiment, a system for lightweight promotion of chronic disease models based on big data is used to execute a method for lightweight promotion of chronic disease models based on big data, including:

[0051] A multi-level federated distillation module that enables medical institutions to collaboratively train base models without sharing raw data;

[0052] A regional adaptive hierarchical learning module is used to group medical institutions based on geographic location and disease spectrum characteristics, and to construct a regional adaptive layer for each group;

[0053] The device resource profiling and model variant generation module is used to profile the terminal devices and classify them based on federated clustering, generating model variants of corresponding complexity for different resource levels;

[0054] Dynamic resource perception module, which monitors the device resource status in real time and selects the model variant that best suits the current resource status based on a scoring function;

[0055] The model update module is used to continuously optimize the model based on local feedback data from various medical institutions through a differential privacy protection mechanism.

[0056] The beneficial effects of the present invention are:

[0057] This paper presents a lightweight, big-data-based generalization method for chronic disease models. Through innovative multi-level federated distillation technology and differential privacy mechanisms, it enables efficient collaboration among medical institutions without sharing raw data, significantly reducing communication overhead while effectively protecting patient privacy. This method overcomes the limitations of traditional federated learning and makes model training more efficient and secure.

[0058] By incorporating regionally adaptive layered learning technology, this paper can automatically identify and adapt to differences in disease spectrum characteristics and treatment patterns across different regions, significantly improving the model's applicability and predictive accuracy across different regions. The design of the regional adaptive layer gives the model excellent transferability, significantly improving cross-regional application effectiveness.

[0059] This invention utilizes model deployment technology based on device resource profiling and dynamic resource awareness to enable flexible model deployment across devices with varying computing power. By generating model variants at multiple complexity levels and dynamically switching between them in conjunction with real-time resource monitoring, this approach ensures both predictive performance and improved resource utilization, enabling the model to run efficiently on a wide range of devices, from high-performance servers to resource-constrained edge devices.

[0060] This invention utilizes a differentially private model update mechanism that enables each medical institution to securely contribute local feedback data for model optimization, ensuring continuous model evolution while protecting data privacy. This incremental optimization approach significantly reduces model maintenance costs, improves update efficiency, and enables the model to promptly adapt to new changes in medical practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a flow chart of a method for lightweight promotion of chronic disease models based on big data of the present invention;

[0062] Figure 2 is a line graph of the model complexity and prediction accuracy of the present invention;

[0063] Figure 3 is a radar chart of performance indicators of the model variants of the present invention on different devices;

[0064] Figure 4It is a bar graph showing the regional adaptability improvement effect of the present invention;

[0065] Figure 5 It is a pie chart of the terminal device type distribution of the present invention. DETAILED DESCRIPTION

[0066] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed solely to enable those skilled in the art to better understand and implement the subject matter described herein, and that the functions and arrangements of the elements discussed may be varied without departing from the scope of this specification. Various examples may omit, substitute, or add various processes or components as needed. Furthermore, features described in some examples may be combined in other examples.

[0067] At least one embodiment of the present invention discloses a lightweight promotion method for chronic disease models based on big data, such as Figure 1 As shown, the following steps are included:

[0068] Step 1: Multi-level federated distillation with differential privacy protection is used to enable medical institutions to collaboratively train the basic model without sharing the original data;

[0069] The specific steps include:

[0070] Step 1.1: Each participating medical institution trains an initial model using local patient data;

[0071] Each institution trains an initial model using its local data. This step is completed internally within each institution, without the need for data sharing. The initial model can be any machine learning model suitable for chronic disease prediction, such as a neural network or gradient boosting tree.

[0072] In some implementations, the initial model can use different architectures and training methods:

[0073] For institutions with larger sample sizes, deep neural network models such as multilayer perceptrons or convolutional neural networks can be used;

[0074] For institutions with moderate sample sizes, ensemble learning methods such as random forests or gradient boosted decision trees can be used;

[0075] For institutions with smaller sample sizes, regularized logistic regression or support vector machine models can be used.

[0076] Institutions can also further improve the quality of initial models by performing model selection and hyperparameter optimization based on local data characteristics. For example, cross-validation methods can be used to automatically select the model type and parameter configuration that best suits the local data distribution.

[0077] Step 1.2: Generate model output statistics for differential privacy protection.

[0078] Institutions do not directly share model parameters or gradients. Instead, they use a public test dataset (anonymous data without sensitive information) to generate statistical information about the model output and add noise that satisfies differential privacy.

[0079] The output statistics here refer to the model's prediction results for public data plus random noise that conforms to a specific distribution, thereby protecting the privacy of the original data.

[0080] The noise parameters are determined based on the preset privacy budget and data sensitivity. The privacy budget parameter controls the strength of privacy protection, and the data sensitivity indicates the maximum impact of a single sample on the output result.

[0081] O i =f i (X pub )+N(0,σ 2 );

[0082] Among them, O i represents the output statistics with differential privacy protection generated by the i-th medical institution; f i represents the local model of the i-th medical institution; X pub represents the public test dataset; N(0, σ 2 ) means the mean is 0 and the variance is σ 2 Gaussian noise; σ 2 Represents the noise variance, which is used to control the strength of privacy protection.

[0083] In some implementations, an adaptive differential privacy protection strategy can be employed to dynamically adjust the privacy budget based on the data sensitivity level. For example, stricter privacy protection can be applied to prediction results containing highly sensitive information (such as mental illnesses), while less sensitive information (such as common chronic disease risk scores) can be protected with relatively looser privacy protection.

[0084] In addition to the Gaussian noise mechanism, other noise addition methods that meet differential privacy can also be used, such as the Laplace mechanism or the exponential mechanism, and the most suitable privacy protection method can be selected according to the characteristics of the output data.

[0085] Step 1.3: The central server integrates the model output information of each institution and constructs a global teacher model;

[0086] The central server collects the output statistics of differential privacy protection generated by each institution and integrates this information using a weighted average method.

[0087] The weight distribution can be determined based on the data volume, data quality or model performance of each institution, and the condition that the total weight is 1 is met.

[0088] In some implementations, weight allocation can adopt multiple strategies:

[0089] Weight based on data volume: The weight is the proportion of each institution's data volume to the total data volume;

[0090] Weight based on model performance: The weight is the ratio of the accuracy of each institution's model on the validation set to the total accuracy;

[0091] Weights based on medical institution level: Different weights are assigned according to the level of the medical institution (such as tertiary, tertiary, etc.);

[0092] Mixed weight strategy: A weighted combination that comprehensively considers the above factors.

[0093] The central server can also implement anomaly detection mechanisms to identify and filter out model outputs that may contain malicious perturbations, thereby improving the robustness of the global model. For example, by calculating the distance between each institution's output and the average output, anomalous outputs with excessively large distances can be eliminated.

[0094] Step 1.4: Transfer global knowledge to the lightweight student model through knowledge distillation technology;

[0095] The central server uses the integrated global output statistics as the output of the teacher model to train a student model with fewer parameters and lower computational complexity.

[0096] The core loss function of knowledge distillation is retained here because it directly expresses the core technical points of this application:

[0097] L distill =α distill ·L CE (f S (X pub ),Y pub )+(1-α distill )·L KL (f S (X pub ),O global );

[0098] Among them, L distill represents the total loss function of knowledge distillation; α distill Represents the weight parameter for balancing the two losses, with a value range of [0, 1]; L CE represents the cross entropy loss function; f S represents the student model; X pub represents the input features of the public dataset; Y pub represents the true label of the public dataset; L KL represents the KL divergence loss function; Oglobal Represents the output distribution of the teacher model (global model) on the public dataset.

[0099] The first item L CE (fS(X pub ), Y pub ) is the cross entropy loss function, which is used to learn the true labels of the public dataset to ensure that the student model can correctly classify. KL (f S (X pub ), O global ) is the KL divergence loss function, which is used to learn the output distribution of the teacher model so that the student model can imitate the behavior of the teacher model. distill Used to balance the importance of these two losses.

[0100] In some implementations, the knowledge distillation process can employ a variety of technical variations:

[0101] Temperature-regulated distillation: introducing a temperature parameter to soften the output distribution of the teacher model;

[0102] Feature-level distillation: In addition to the final output, intermediate-level features are also distilled.

[0103] The structure of the student model can be customized based on the computing power of the target deployment environment. For example, for medium-computing environments, efficient structures such as depthwise separable convolutions can be selected; for low-computing environments, more streamlined network structures such as MobileNet or ShuffleNet can be selected.

[0104] Step 1.5: Distribute the lightweight basic model to each medical institution;

[0105] The central server distributes the trained lightweight student model as the base model to each participating institution as the basis for subsequent steps.

[0106] This lightweight base model incorporates the combined knowledge of all participating institutions, but with significantly reduced model parameters, making it suitable for subsequent further optimization and deployment.

[0107] In some implementations, model distribution can adopt different strategies:

[0108] Version control distribution: Assign a version number to each model update, so each organization can choose when to update based on its own situation;

[0109] Incremental distribution: only the changes in model parameters are transmitted to reduce communication overhead;

[0110] On-demand distribution: Distribute model versions of specific accuracy or size based on the specific needs of each institution.

[0111] The distribution process can include a model validation step to ensure that the model performance received by each institution meets expectations. For example, each institution can test the received model on a local validation set and feed the validation results back to the central server to monitor the model's performance in different environments.

[0112] Through the above steps, the method provided in this application solves the problems of high communication overhead and insufficient privacy protection in traditional federated learning. It realizes the collaborative training of high-quality chronic disease prediction models using data distributed across various medical institutions while strictly protecting patient privacy, and achieves the initial lightweighting of the model through knowledge distillation technology.

[0113] Step 2: Based on the basic model, medical institutions are grouped according to their geographical location and disease spectrum characteristics. A regional adaptive layer is constructed for each group to achieve regional adaptability of the model and obtain a regional adaptive model.

[0114] The specific steps include:

[0115] Step 2.1: Group medical institutions based on geographic location and disease spectrum characteristics;

[0116] Based on information such as the geographical location of the medical institutions, characteristics of the population they serve, and distribution of disease spectra, all participating institutions were divided into multiple groups, each of which contained institutions with similar disease spectra and population characteristics.

[0117] In addition, grouping can be based on administrative divisions (such as provinces and cities) or medical resource distribution characteristics (such as urban agglomerations and medical alliances), or it can be automatically generated through clustering algorithms.

[0118] In some embodiments, the grouping method can employ a variety of strategies:

[0119] Hard division based on geographical location: grouping directly by administrative division or geographical distance;

[0120] Soft segmentation based on disease spectrum similarity: Calculate the similarity matrix of disease spectrum distribution of each institution and use spectral clustering or hierarchical clustering algorithm to group them;

[0121] Hybrid classification method: considers both geographical location and disease spectrum characteristics, and uses a multi-feature clustering algorithm for grouping.

[0122] The grouping process can be dynamic. As new data accumulates, the system can periodically re-evaluate the grouping of each institution and adjust the group divisions when necessary to adapt to changing trends in the disease spectrum.

[0123] Step 2.2: Conduct local federated learning within each group to build a regional specialized model.

[0124] For each group, the medical institutions within it use the lightweight base model distributed in step 1 as the initial model, fine-tune it based on their respective local data, and then use the federated averaging algorithm to generate a group-specific model.

[0125] The federated averaging algorithm generates a group model by weighted averaging of the model parameters of each institution, where the weights are usually proportional to the amount of data from each institution.

[0126] In some implementations, local federated learning can employ different aggregation methods:

[0127] Parameter weighted averaging: as described above for the standard federated averaging algorithm;

[0128] Adaptive aggregation: Dynamically adjust weights based on the performance of each model on a shared validation set;

[0129] Robust aggregation: pruning or scaling abnormal parameters to reduce the impact of extreme values;

[0130] Hierarchical aggregation: first aggregate in a small range, then gradually expand the aggregation range to form a hierarchical aggregation process.

[0131] Optionally, if resources are sufficient, regularization constraints can be applied to the local federated learning process, such as adding a consistency regularization term, so that the group-specialized model maintains a certain functional similarity with the basic model, avoiding over-adaptation to local data and loss of general knowledge.

[0132] Step 2.3: Exchange common knowledge between groups through cross-regional knowledge distillation;

[0133] In order to enable each group model to learn from each other while maintaining their respective regional characteristics, this application uses cross-regional knowledge distillation technology to exchange knowledge between groups.

[0134] During this process, the KL divergence between the prediction results of each group model on the public test data is calculated as the loss function. By minimizing this loss function, each group model can learn valuable knowledge from each other.

[0135] The weight parameter represents the degree of knowledge contribution between groups and can be determined based on the similarity of disease spectra between groups.

[0136]

[0137] Among them, L cross represents the loss function of cross-region knowledge distillation; K group Represents the total number of groups; i and j represent the indexes of different groups respectively; represents the knowledge exchange weight between groups i and j; represents the local model of the i-th group; represents the local model of the jth group; X pub represents the public test data set; KL represents the KL divergence, which is used to measure the difference between two probability distributions.

[0138] In some implementations, inter-group knowledge exchange can employ different methods:

[0139] Bidirectional knowledge distillation: As in the above method, each group learns from each other;

[0140] Hierarchical knowledge distillation: Establish a hierarchical structure between groups and transfer knowledge from the upper group to the lower group;

[0141] Selective knowledge distillation: each group learns only from the few groups that are most similar to it;

[0142] Dynamic weighted knowledge distillation: Dynamically adjust weight values ​​based on performance during the distillation process.

[0143] Knowledge exchange between groups can also adopt a joint training method, which can enhance the general representation learning ability while retaining the group specialization ability by sharing some network layers or feature extractors.

[0144] Step 2.4: construct the regional adaptation layer to realize the regional adaptation capability of the model;

[0145] On the basis of the basic model, a lightweight regional adaptation layer is added for each group to form a complete regional adaptation model.

[0146] This structure allows the model to have both a universal base part and a region-specific adaptation part. The region-specific coefficient is used to control the degree of region specialization. The region-specific layer is designed as a lightweight structure, such as a single-layer neural network, which only requires a small number of parameters to capture region-specific features.

[0147] In some implementations, the region adaptation layer can adopt different structures:

[0148] Residual Adaptation Structure: The regional specialized knowledge is added as the residual term of the basic model output to the final output;

[0149] Feature conversion structure: Use region-specific feature conversion functions to process the input and then pass it to the base model;

[0150] Attention modulation architecture: Uses region-specific attention maps to modulate the output of the base model;

[0151] Conditional Normalization Structure: Introducing region-specific scaling and offset parameters in the normalization layer of the base model.

[0152] The regional specialization coefficient can be designed as a dynamic parameter, automatically adjusting the degree of specialization based on the characteristics of the input data. For example, for common cases with similar manifestations across multiple regions, the specialization coefficient can be reduced, relying more on the basic model; for unique cases with significant regional differences, the specialization coefficient can be increased, relying more on the regional specialization layer.

[0153] Step 2.5: Generate a lightweight model for each participating institution that is suitable for its regional characteristics;

[0154] The central server combines the base model with the corresponding regional adaptation layer to generate a lightweight model for each institution that is tailored to its specific region. This model structure maintains the lightweight nature of the core while providing adaptability to different regions through a small number of regionally specific parameters.

[0155] In some embodiments, the model generation process can take into account specific characteristics of the institution:

[0156] For large medical groups with multiple branches, a comprehensive model with multiple regional adaptive layers can be generated;

[0157] For institutions located at the junction of regions, a hybrid model that fuses specialized layers from multiple adjacent regions can be generated;

[0158] For specialized hospitals, the model parts related to their specialties can be further enhanced.

[0159] In addition to the regional adaptation layer, specialized adaptation modules can be added to address institutional characteristics. For example, an age-sensitive feature processing unit can be added to a pediatric hospital, or the analysis capabilities of cancer-related risk factors can be enhanced for an oncology hospital.

[0160] Through the above steps, the method provided in this application solves the problem of model migration difficulties caused by differences in disease spectra and population characteristics in different regions, and enables the model to adapt to the characteristics of different regions while maintaining its lightweight, thereby improving the prediction accuracy and practicality of the model in various regions.

[0161] Step 3: Based on the regional adaptability model, we perform resource profiling on the terminal devices and classify them based on federated clustering. We generate model variants of corresponding complexity for different resource levels and ensure knowledge transfer between model variants through progressive knowledge distillation.

[0162] The specific steps include:

[0163] Step 3.1: Collect resource information of each terminal device in the medical institution to form a resource profile;

[0164] Resource information is collected for various terminal devices in each medical institution, including processor performance (CPU type, number of cores, main frequency, etc.), memory capacity, storage space, power status (battery capacity, whether it is connected to the power supply for a long time, etc.) and network connection (bandwidth, stability, etc.).

[0165] Integrate this information into a device resource profile:

[0166] P device ={cpu, mem, power, network};

[0167] Among them, P device Represents the device resource profile, which is used to describe the overall resource status of the device; cpu represents processor performance indicators, including parameters such as CPU type, number of cores, main frequency, cache size, etc.; mem represents memory capacity indicators, including parameters such as total memory capacity, available capacity, and memory speed; power represents power status indicators, including parameters such as battery capacity, remaining power, whether an external power supply is connected, and power consumption; network represents network connection indicators, including parameters such as network type, bandwidth, latency, packet loss rate, and signal strength.

[0168] Each indicator is in the form of a multidimensional vector and can contain multiple specific parameter values ​​to comprehensively describe the resource characteristics of the device.

[0169] Step 3.2: Use the federated clustering algorithm to classify the device resource profiles;

[0170] Based on the device resource profile, the federated K-means clustering algorithm is used to classify all devices into multiple resource levels:

[0171]

[0172] Among them, FedKMeans represents the federated clustering algorithm, which is used to classify devices while protecting privacy; Represents the 1st, 2nd, and nth device Resource profile of each device; n device represents the total number of devices; C represents the clustering result, that is, the resource level category to which each device is assigned.

[0173] Federated clustering algorithms allow for collaborative clustering without sharing original device information, protecting device privacy. Devices are typically categorized into three primary resource levels: high, medium, and low, or more fine-grained based on actual conditions.

[0174] The federated clustering algorithm in this embodiment is specifically implemented as follows:

[0175] Initialization: The central server randomly generates K cluster Cluster centers And distributed to various medical institutions.

[0176] Local clustering: Each medical institution uses the received cluster centers to cluster and allocate local device resource profiles.

[0177] For each device resource profile Calculate the distance from each cluster center and assign it to the nearest cluster center:

[0178]

[0179] in, Indicates the cluster center number to which the resource profile of the j-th device is assigned; Indicates the resource profile of the jth device; argmin k Indicates the number of the cluster center k that minimizes the distance; dist is the distance function used to calculate the distance between the device resource profile and the cluster center, which can be Euclidean distance or cosine similarity; c k represents the kth cluster center.

[0180] Local update: Each medical institution calculates the average value of the device resource profiles locally assigned to each cluster center and uses it as the new local cluster center:

[0181]

[0182] in, Represents the local update value of the k-th cluster center; Represents the resource profile of the jth device; Indicates the cluster center number to which the jth device is assigned; k indicates the currently calculated cluster center number; Indicates the sum of all devices assigned to the k-th cluster center; Indicates the number of devices assigned to the kth cluster center.

[0183] To protect privacy, noise that satisfies differential privacy is added to the local cluster center:

[0184]

[0185] in, represents the kth local cluster center after adding differential privacy noise; represents the original k-th local cluster center; N(0, σ 2 ) means the mean is 0 and the variance is σ 2 Gaussian noise; σ 2 Represents the noise variance, which is used to control the privacy protection strength. The larger the value, the higher the privacy protection.

[0186] Global aggregation: Each medical institution will cluster its local centers The data is sent to the central server, which performs weighted averaging to generate a new global cluster center:

[0187]

[0188] in, represents the new global update value of the kth cluster center; n device Indicates the total number of healthcare institutions participating in federated learning; represents the weight of the i-th medical institution, which can be determined according to the number of its equipment; represents the kth local cluster center after adding differential privacy noise to the i-th medical institution; Indicates the sum of all medical institutions.

[0189] Iterative optimization: Repeat the steps until the cluster centers become stable or the preset number of iterations is reached.

[0190] Final classification: Use the final cluster center to classify all devices and obtain the device resource level division result C.

[0191] In chronic disease management systems, federated clustering algorithms are applied to classify equipment resources across multiple levels of medical institutions. For example, within a provincial chronic disease management network, this algorithm can automatically categorize medical equipment across the province into three resource tiers: high-performance servers, standard workstations, and mobile terminals, while protecting the privacy of equipment information at different levels of hospitals. This provides a foundation for subsequent model variant generation.

[0192] Step 3.3, generate corresponding model variants for each resource level;

[0193] Based on the regional adaptability model generated in step 2, a model variant with corresponding complexity is generated for each resource level:

[0194] {M high , M medium , M low}=ModelVariantGen(M region , C1);

[0195] Among them, ModelVariantGen represents the model variant generation algorithm, which is used to generate model variants of different complexities; M region represents the regional adaptability model, that is, the original model adjusted according to regional characteristics; C1 represents the device resource level division result, which includes the resource level information allocated to each device; M high represents the generated high-complexity model variant, suitable for devices with sufficient computing resources; M mediumrepresents the generated medium-complexity model variant, suitable for devices with moderate computing resources; M low Represents a generated low-complexity model variant suitable for devices with limited computational resources.

[0196] For medium-resource tier devices, a moderately compressed model variant can be used;

[0197] For low-resource tier devices, a highly compressed model variant is required.

[0198] The model variant generation algorithm in this embodiment is specifically implemented as follows:

[0199] High-complexity model variant generation: For high-resource-level devices, directly use the regional adaptive model M region As a high complexity model variant M high , retaining all parameters and structure of the model to provide the highest prediction accuracy.

[0200] Medium complexity model variant generation: For medium resource level devices, network pruning and selective quantization techniques are used to generate medium complexity model variants M medium :

[0201] Importance analysis: calculate the importance index of each layer parameter of the model l , which can be evaluated by parameter gradient amplitude or activation value variance;

[0202] Parameter pruning: Based on the importance index, the top 70% of the parameters are retained and the remaining parameters are set to zero;

[0203] Selective quantization: Different quantization strategies are used for different layers. Key layers (such as the output layer) maintain high precision, and intermediate layers are quantized with 8-bit precision.

[0204] Low-complexity model variant generation: For low-resource level devices, model distillation and structure reorganization techniques are used to generate low-complexity model variants M low :

[0205] Core structure extraction: Analyze the original model and extract key predictive units;

[0206] Layer fusion: Merge multiple consecutive layers into a single computing unit to reduce the storage of intermediate results;

[0207] Knowledge distillation: training a small network with significantly reduced parameters to mimic the output behavior of the original model;

[0208] Variant adaptation optimization: Further optimize each model variant based on the hardware characteristics of the target device:

[0209] Computational graph optimization: adjust the order of operations to maximize computational parallelism;

[0210] Memory access optimization: reorganize data structures to improve cache hit rate;

[0211] Specialized instruction set adaptation: Utilize special instructions of the target hardware (such as SIMD, neural network acceleration instructions, etc.).

[0212] Step 3.4: Ensure knowledge transfer between model variants of different complexity through progressive knowledge distillation technology;

[0213] A progressive knowledge distillation strategy is used to transfer knowledge from high-complexity models to low-complexity models:

[0214] L progressive =KL(f high (x)||f medium (x))+KL(f medium (x)||f low (x));

[0215] Among them, L progressive is the progressive knowledge distillation loss function, which is used to measure the knowledge transfer effect between models of different complexity; KL represents the KL divergence, which is used to measure the difference between two probability distributions; f high (x) represents the predicted output probability distribution of the high-complexity model for input x; f medium (x) represents the predicted output probability distribution of the medium complexity model for input x; f low (x) represents the predicted output probability distribution of the low-complexity model for input x; x represents the input data sample.

[0216] The progressive knowledge distillation technology in this embodiment is specifically implemented as follows:

[0217] Distillation data preparation: building knowledge distillation training set D distill , contains two parts of data:

[0218] Unlabeled public data: samples related to the target task but without sensitive information are selected from public medical datasets;

[0219] Synthetic data: using a high complexity model M high Generate simulated data that mimics the distribution characteristics of real medical data;

[0220] Medium complexity model distillation:

[0221] Using a high complexity model M high For the distillation dataset D distill Make predictions and get soft labels:

[0222] S high ={fhigh (x i )|x i ∈D distill};

[0223] Among them, M high Indicates a teacher model with high complexity; D distill represents the training dataset for knowledge distillation; f high Represents the prediction function of a high-complexity model; x i represents the i-th sample in the data set; S high Represents the set of soft labels generated by the high-complexity model.

[0224] Initialize the medium complexity model M medium , and trained using the following loss function:

[0225]

[0226] Among them, L emdium is the total loss function of the medium complexity model; α distill is a balance parameter used to adjust the proportion of true label loss and distillation loss; L CE is the cross entropy loss function; f medium (x) is the predicted output of the medium complexity model for input x; y is the true label; T temp is the temperature parameter, set to 2 to 5, used to soften the probability distribution; KL is the KL divergence, used to measure the difference between two probability distributions; f high (x) is the predicted output of the high-complexity model for input x.

[0227] During the training process, gradually increase α distill value, so that the model gradually shifts from imitating the teacher model to learning the true label.

[0228] Low-complexity model distillation:

[0229] Use the trained medium complexity model M medium Generate new soft labels for the distilled dataset:

[0230] S medium ={f medium (x i )|x i ∈D distill};

[0231] Among them, M medium represents the trained teacher model of medium complexity; D distill represents the training dataset for knowledge distillation; f medium represents the prediction function of the medium complexity model; x irepresents the i-th sample in the data set; S medium Represents the set of soft labels generated by a medium-complexity model.

[0232] Initialize the low complexity model M low , and trained using the following loss function:

[0233]

[0234] Among them, L low is the total loss function of the low-complexity model; β distill is a balance parameter, usually smaller than α distill , used to adjust the proportion of true label loss and distillation loss; L CE is the cross entropy loss function; f low (x) is the predicted output of the low-complexity model for input x; y is the true label; T temp is the temperature parameter, which is used to soften the probability distribution; KL is the KL divergence, which is used to measure the difference between two probability distributions; f medium (x) is the predicted output of the medium complexity model for input x.

[0235] Intermediate feature distillation: In addition to the final output, the intermediate features of the model are also distilled:

[0236] Select the output of the key layer as the auxiliary distillation target in the medium complexity model;

[0237] Set the corresponding feature conversion function in the student model to map the intermediate features of the student model to the same dimension as the teacher model;

[0238] Add feature matching loss:

[0239]

[0240] Among them, L feature is the feature matching loss function; Indicates the sum of all layers, L is the total number of layers; γ layer is the weight coefficient of each layer feature, which is used to adjust the importance of different layer features; g layer is the feature conversion function, which maps the student model features to the teacher model feature space; It is the feature output of the layer layer of the student model; It is the feature output of the layer layer of the teacher model.

[0241] Step 3.5, perform special optimization for low-resource device model variants;

[0242] For devices with extremely limited resources (such as wearable devices and home health monitoring devices), in addition to general model compression, special optimization techniques are also required:

[0243] Model quantization: Convert model parameters from 32-bit floating-point numbers to 8-bit integers or lower bit width, significantly reducing storage requirements and computational complexity.

[0244] Structural pruning: Remove connections and neurons with less contribution in the model to reduce the number of model parameters and computational complexity.

[0245] Low-rank decomposition: Decompose a large matrix into the product of multiple small matrices to reduce the number of parameters and computational complexity.

[0246] Hardware-aware optimization: Specific optimizations are made to the model structure and calculation process based on the hardware characteristics of the target device (such as SIMD instruction set, cache size, etc.).

[0247] Through the above steps, this method solves the problem of computing resource diversity and realizes adaptive deployment of models for devices with different computing capabilities, enabling the chronic disease prediction model to run efficiently on various terminals, from high-performance servers to resource-constrained edge devices, greatly expanding the application scope of the model.

[0248] Step 4: Based on the generated model variants, monitor the device resource status in real time and select the model variant version that best suits the current resource status based on the scoring function to achieve dynamic deployment;

[0249] The specific steps include:

[0250] Step 4.1: Monitor the device resource status in real time and update the resource profile;

[0251] Run a lightweight resource monitoring module on the device to collect real-time device operating status information, including current CPU usage, available memory, remaining battery power, network connection status, etc., and update the device's resource profile:

[0252] P current =ResourceMonitor(device);

[0253] Among them, P current Represents the resource profile of the current device, a vector containing resource status information such as CPU usage, available memory, and battery power. ResourceMonitor represents the resource monitoring module, a function used to collect and analyze device resource usage. Device represents the terminal device to be monitored, which can be a doctor's workstation, mobile ward rounds equipment, or patient self-service terminal. The frequency of resource monitoring can be flexibly adjusted according to the device type and application scenario to balance monitoring accuracy and monitoring overhead.

[0254] In some implementations, resource monitoring can employ a variety of strategies:

[0255] Periodic monitoring: collecting resource information at fixed time intervals;

[0256] Event-triggered monitoring: The resource profile is updated only when the resource usage rate exceeds the threshold.

[0257] Predictive monitoring: Combine historical data to predict future resource status and make adjustments in advance;

[0258] Hierarchical monitoring: core resources (such as CPU and memory) are monitored at high frequency, and secondary resources are monitored at low frequency.

[0259] Resource monitoring can also take application scenarios into consideration. For example, in an emergency setting, the weight of battery consumption can be reduced to prioritize model performance; in a home monitoring environment, the weight of battery life can be increased to extend device usage time.

[0260] Step 4.2, select the most suitable model variant based on the current resource status;

[0261] Based on the resource profile of the current device, select the version that best suits the current state from the multiple model variants generated in step 3:

[0262]

[0263] Among them, M * represents the optimal model variant finally selected; M high Indicates a high-complexity model variant; M medium Indicates a medium complexity model variant; M low Represents a low-complexity model variant; Score is a scoring function used to evaluate the model variant M in the current resource state P current Adaptability under P current Represents the resource status vector of the current device, including information such as CPU usage and available memory; argmax represents the model variant that maximizes the scoring function Score.

[0264] The scoring function takes into account model performance and resource consumption and can be expressed as:

[0265] Score(M,P)=α score Accuracy(M)-β score ResourceCost(M,P);

[0266] Where M represents the model variant to be evaluated, and P represents the resource state vector of the current device; Score(M, P) represents the comprehensive score of the model variant M under the resource state P; Accuracy(M) represents the prediction accuracy of the model variant M, with a value range of [0, 1]; ResourceCost(M, P) represents the resource consumption cost of the model variant M under the resource state P, including execution time, memory usage, energy consumption, etc.; α score The weight coefficient of accuracy is used to control the importance of accuracy in scoring; β score Represents the weight coefficient of resource consumption, which is used to control the importance of resource consumption in the score; α score and β score The value of needs to be adjusted according to the actual application scenario, and the sum of the two is usually normalized to 1.

[0267] Step 4.3, realize smooth switching between model variants;

[0268] When switching from one model variant to another, a smooth switching strategy is adopted to avoid sudden performance fluctuations:

[0269] f transition (x) = (1-t transition )·f old (x)+t transition ·f new (x);

[0270] Among them, f transition (x) represents the output result during the model switching transition, which is a weighted combination function; x represents the input data sample; f old (x) represents the predicted output of the old model variant for input x; f new (x) represents the predicted output of the new model variant for input x; t transition is a smoothing factor with a value range of [0, 1], which is used to control the weight of the output of the new and old models;

[0271] When t transition =0, the output of the old model is used completely;

[0272] When t transition =1, the output of the new model is fully used;

[0273] When 0<t transition When <1, the outputs of the new and old models are mixed according to the weights.

[0274] This smooth switching mechanism gradually adjusts t transition , achieving a smooth transition between the outputs of the old and new models can effectively reduce the disturbance of model variant switching on the prediction results and ensure system stability.

[0275] Step 4.4: Implement an incremental update protocol to support efficient updates of model variants.

[0276] Design a lightweight incremental update protocol to enable each model variant to share the benefits of global model updates:

[0277] Update incre (M i , ΔM)=M i +α update ΔM;

[0278] Among them, Update incre Indicates the incremental update operation function, which is used to incrementally update the model; M i represents the original model variant to be updated; ΔM represents the model parameter update amount obtained from the central server; α update is the adaptation factor, ranging from [0, 1], used to control the intensity of the update; M i +α update ·ΔM represents the original model M i With α update The adjusted updates are added together to obtain the updated model.

[0279] When α update =1, the update amount is fully applied;

[0280] When α update =0, no update is applied;

[0281] When 0<α update When < 1, partial update amount is applied proportionally.

[0282] Different adaptation strategies are adopted for model variants of different complexity: high-complexity models can directly apply complete updates, while medium and low-complexity models need to be transformed and updated through knowledge distillation or parameter mapping.

[0283] Step 4.5: Dynamically adjust model behavior based on user interaction and task importance;

[0284] Dynamically adjust the model's working mode based on different user interaction patterns and task importance:

[0285] For critical tasks (such as high-risk patient monitoring), priority is given to ensuring prediction accuracy even in resource-constrained situations, and switching to higher-complexity model variants when necessary.

[0286] For routine tasks (such as regular health assessments), when resources are tight, the model complexity can be appropriately reduced to save computing resources.

[0287] During active user interaction, the model complexity can be temporarily increased to provide more accurate prediction results; when running in the background, it switches to a more energy-efficient lightweight model variant.

[0288] Through the above steps, this method realizes the adaptive deployment of chronic disease prediction models on devices with dynamically changing resources. It can dynamically select the most suitable model variant based on the current resource status of the device, user interaction mode and task importance, maximize resource utilization efficiency while ensuring prediction performance, and improve the adaptability of the model in practical applications and user experience.

[0289] Step 5: Based on the deployed model variants, local feedback data from each medical institution is collected through a differentially private model update mechanism to achieve continuous model optimization.

[0290] The specific steps include:

[0291] Step 5.1, each medical institution collects local prediction feedback data;

[0292] When using the chronic disease prediction model, each institution collects comparative data between the predicted results and the actual results to form a feedback data set F i .

[0293] These data include cases where the forecasts were accurate and off, and especially cases where the forecasts were wrong but representative.

[0294] Step 5.2: Use local feedback data to fine-tune the model;

[0295] Each institution uses local feedback data F i For the current model M i Fine-tune and generate an improved local model M i ′:

[0296] M i ′=Finetune(M i , F i );

[0297] Among them, M i ′ represents the fine-tuned local model; M i represents the original local model; F i Represents the locally collected feedback dataset; Finetune represents the model fine-tuning function, which is used to locally optimize the model based on the feedback data.

[0298] The fine-tuning process uses a gradient descent algorithm with a small learning rate to avoid overfitting local data and losing the versatility of the original model.

[0299] Step 5.3: extract the knowledge increment of local model improvement;

[0300] To avoid directly sharing model parameters and reduce communication overhead, each institution extracts knowledge increments for local model improvements:

[0301] ΔK i =M i ′(X pub )-M i (X pub );

[0302] Where ΔK i represents the knowledge increment of the i-th medical institution; M i ′ represents the model after local fine-tuning; M i represents the original local model; X pub represents the public test dataset; M i ′(X pub ) represents the predicted output of the fine-tuned model on the public test dataset; Mi(X pub ) represents the prediction output of the original model on the public test dataset; the difference between the two reflects the prediction change of the model before and after fine-tuning, that is, the knowledge increment.

[0303] To protect privacy, noise that satisfies differential privacy is added to the knowledge increment:

[0304]

[0305] in, represents the knowledge increment after adding differential privacy noise; ΔK i represents the original knowledge increment; N(0, σ 2 ) means the mean is 0 and the variance is σ 2 Gaussian noise; σ 2 Represents the noise intensity parameter, which is used to control the degree of privacy protection.

[0306] Step 5.4: The central server aggregates the knowledge increments and generates global model updates;

[0307] The central server collects the knowledge increments submitted by each institution and aggregates them using a weighted average method:

[0308]

[0309] Where ΔK global represents the global knowledge increment after aggregation; n represents the total number of medical institutions participating in the update; represents the updated weight coefficient of the i-th medical institution; represents the knowledge increment provided by the i-th medical institution after adding differential privacy protection; Represents the weighted sum of the knowledge increments of all n medical institutions.

[0310] Step 5.5: Convert the global knowledge increment into model update and send it to each terminal;

[0311] The central server converts the global knowledge increment into model parameter update ΔM global , and sent to each agency, which applies it to the current model variant according to the incremental update protocol.

[0312] Step 5.6: Establish update cycle and emergency update mechanism;

[0313] Based on the characteristics of chronic disease prediction models, a two-level update mechanism is established:

[0314] Regular cycle updates: Model updates are performed regularly (e.g. monthly or quarterly);

[0315] Emergency update mechanism: activated when systematic prediction deviations or major updates in medical knowledge are discovered.

[0316] Through the above steps, this method achieves the continuous optimization and updating of the chronic disease prediction model, enabling the model to continuously incorporate new medical knowledge and data features, maintaining the model's advancedness and applicability, and avoiding the problem of gradual performance degradation after deployment of traditional models.

[0317] Application examples of this implementation:

[0318] In order to demonstrate the application effect of the lightweight promotion method of chronic disease model based on big data provided by this application in actual scenarios, a simulated application example is given below, which is based on the promotion and application of diabetes risk prediction model in regional medical network.

[0319] Application scenarios:

[0320] This example uses a regional medical network consisting of one tertiary hospital, four secondary hospitals, and 15 community medical centers. This network covers the city center and surrounding suburbs, serving a population of approximately 2 million. The data scale and computing resources of each medical institution vary significantly:

[0321] The tertiary hospital has complete electronic health records for approximately 100,000 patients with diabetes and prediabetes and is equipped with a high-performance computing cluster;

[0322] Secondary hospitals each have electronic health records of 10,000 to 31,000 patients and are equipped with servers of medium performance;

[0323] Community health centers each hold partial electronic health records for 3,000 to 8,000 patients and have only basic computing equipment;

[0324] Patient data across medical institutions cannot be directly shared and exhibit significant regional differences:

[0325] Patients in urban centers are mainly office workers, with a higher proportion of hyperlipidemia and hypertension;

[0326] Suburban patients are mainly manual laborers, with obvious differences in dietary structure and living habits.

[0327] In addition, the medical network has diverse equipment resources, ranging from high-performance servers to doctor workstations, mobile ward rounds equipment, and patient home monitoring equipment, with a wide range of computing capabilities.

[0328] In this application scenario, it is necessary to build a model that can accurately predict the risk of diabetes and disease progression, and deploy it on various terminal devices in the above-mentioned multi-level medical institutions to assist medical decision-making and patient health management.

[0329] Implementation process:

[0330] According to the method of this application, the specific process implemented in the above application scenario is as follows:

[0331] Multi-level federated distillation and privacy protection implementation:

[0332] Each medical institution trains an initial diabetes risk prediction model based on local data:

[0333] The tertiary hospital used a deep learning model consisting of a 6-layer fully connected network;

[0334] The gradient boosting tree model was used in the secondary hospital;

[0335] Logistic regression models with regularization were used for community health centers;

[0336] An anonymous public test dataset was constructed, containing 500 samples without patient identity information. Each sample contained 22 features including routine examination indicators such as age, gender, BMI, blood pressure, blood sugar, and blood lipids.

[0337] Each medical institution generates differentially private model output:

[0338] Institutions applied initial models to public test datasets;

[0339] Set the differential privacy parameter ε = 2.5 according to the data sensitivity;

[0340] Add Gaussian noise to generate privacy-preserving output statistics;

[0341] The central server integrates the model outputs and builds a global teacher model:

[0342] The weight of tertiary hospitals is 0.4, the weight of four secondary hospitals is 0.1 each, and the weight of other medical institutions is 0.2 in total;

[0343] Integrate the differentially private outputs of each institution to generate a global knowledge representation;

[0344] Training a lightweight student model through knowledge distillation:

[0345] A 3-layer neural network was designed as the student model, with only 15% of the parameters of the original model of the tertiary hospital. Soft label distillation technology with temperature T = 3 was used.

[0346] The weight ratio of cross entropy loss to KL divergence loss is 0.3:0.7;

[0347] Lightweight basic models are distributed to various medical institutions as the basis for subsequent regional adaptive training.

[0348] Regionally Adaptive Tiered Learning Implementation:

[0349] Medical institutions are divided into 3 regional groups:

[0350] Group A: urban center area (1 tertiary hospital, 2 secondary hospitals, 5 community medical centers); Group B: urban suburban area (1 secondary hospital, 6 community medical centers);

[0351] Group C: remote suburban area (1 secondary hospital, 4 community medical centers);

[0352] Conduct local federated learning within each group:

[0353] Each institution uses local data to fine-tune the lightweight base model;

[0354] The parameters are integrated using the data-weighted federated averaging algorithm;

[0355] Generate region-specific models to capture the specific disease characteristics of each region;

[0356] Cross-region knowledge distillation between groups:

[0357] Setting knowledge contribution weights based on the similarity of regional population characteristics;

[0358] The weight between groups A and B is 0.6, the weight between groups B and C is 0.5, and the weight between groups A and C is 0.3; knowledge sharing between regions is achieved by minimizing the cross-region KL divergence loss.

[0359] Construct a region-adaptive layer for each group:

[0360] Design a single-layer residual adaptation structure with 32 neurons;

[0361] The urban core area specialization layer enhances sensitivity to characteristics related to sedentary lifestyle;

[0362] The exurban area specialization layer has enhanced sensitivity to characteristics related to diet structure and physical activity;

[0363] Generate region-specific lightweight models for each medical institution:

[0364] The basic model parameters are unified, while the regional adaptive layer parameters are differentiated;

[0365] Bodies located at the junction of regions receive a model of hybrid adaptive layers.

[0366] Device resource profiling and model variant generation implementation:

[0367] Collect resource information of terminal devices in each medical institution:

[0368] Collected hardware configuration information for 210 devices in a tertiary hospital;

[0369] Collect hardware configuration information for a total of 180 devices in a secondary hospital;

[0370] Collect hardware configuration information for a total of 320 devices in the community medical center;

[0371] Select 100 representative home monitoring devices for resource profiling;

[0372] Device classification using federated clustering algorithm:

[0373] Set parameter K=3 for federated K-means clustering;

[0374] Add noise that satisfies ε=3 differential privacy protection;

[0375] After 6 rounds of iterations, the devices were divided into three resource levels: high, medium, and low.

[0376] Generate corresponding model variants for each resource level:

[0377] High resource level: retain the complete regional adaptability model (about 120K parameters);

[0378] Medium resource level: Through parameter pruning and 8-bit quantization, a medium-complexity model with about 40K parameters is generated;

[0379] Low-resource level: Through model distillation and structural reorganization, a lightweight model with approximately 15K parameters is generated.

[0380] Use progressive knowledge distillation to ensure knowledge transfer between model variants:

[0381] First, distill the high-complexity model to the medium-complexity model with a temperature parameter T = 4;

[0382] Then distill from the medium complexity model to the low complexity model with temperature parameter T=3;

[0383] Add the intermediate layer feature matching loss with a weight of 0.3.

[0384] Special optimizations for low-resource device model variants:

[0385] Quantize the model parameters to 4-bit fixed-point numbers;

[0386] Remove the 25% of connections that contribute the least;

[0387] Specially optimized for instruction level on ARM processors used in home monitoring devices.

[0388] Dynamic resource-aware model deployment implementation:

[0389] Deploy lightweight resource monitoring modules on various terminal devices:

[0390] Monitor CPU usage, memory usage, battery status and network connection;

[0391] High-performance devices sample once every 60 seconds, medium-performance devices sample once every 120 seconds, and low-performance devices use event-triggered sampling.

[0392] Design a scoring function for model variant selection:

[0393] Accuracy weight α = 0.7, resource consumption weight β = 0.3;

[0394] Automatically adjust α=0.6, β=0.4 under high load conditions;

[0395] When the battery power is lower than 30%, α is automatically adjusted to 0.5 and β is automatically adjusted to 0.5.

[0396] Enable smooth switching between model variants:

[0397] Use a linear interpolation transition with a duration of 5 seconds;

[0398] Set the resource status to exceed the threshold for 30 seconds before triggering the switch;

[0399] Compare the results of the two models and trigger anomaly detection when the difference exceeds the threshold.

[0400] Deploy incremental update protocol:

[0401] Design differentiation adaptation factor: high complexity model α = 1.0, medium complexity model α = 0.8, low complexity model α = 0.6;

[0402] Achieve compressed transmission of model parameters, reducing the amount of update data by 80%;

[0403] Establish a dual-track mechanism of regular updates and emergency updates once a week.

[0404] Dynamically adjust model behavior based on medical scenarios:

[0405] High-risk patient monitoring is set as a critical task, with accuracy being prioritized;

[0406] Regular health assessments are set as routine tasks to reduce model complexity when resources are limited;

[0407] Temporarily increase the model complexity during doctor interaction to provide more accurate prediction results.

[0408] Technical effect verification:

[0409] This application example fully validated the method provided in this application through a six-month actual deployment, focusing on two core technical effects: "balancing lightweight and accuracy" and "improving cross-regional adaptability":

[0410] Balance between lightness and accuracy:

[0411] In this application, multi-level federated distillation and progressive model compression were used to significantly reduce the model weight while maintaining a high level of prediction accuracy:

[0412] Model size reduction: The original model has 800K parameters, while the lightweight high-, medium-, and low-resource-level model parameters are 120K, 40K, and 15K, respectively, with reductions of 85%, 95%, and 98%, respectively.

[0413] Reduced computational complexity: The number of floating-point operations is reduced from 15 million in the original model to 2.5 million, 800,000, and 300,000 in the high, medium, and low resource tier models, representing reductions of 83%, 95%, and 98%, respectively.

[0414] Reduced memory usage: Runtime memory usage is reduced from 120MB for the original model to 35MB, 12MB, and 4.5MB for the high, medium, and low resource tier models, representing a reduction of 71%, 90%, and 96%, respectively.

[0415] Reduced inference latency: On a standard test device, inference latency is reduced from 320ms for the original model to 85ms, 42ms, and 18ms for the high, medium, and low resource tier models, representing reductions of 73%, 87%, and 94%, respectively.

[0416] Prediction accuracy is maintained: Compared to the original model trained on a single center, the lightweight high-, medium-, and low-resource models achieved AUC values ​​of 0.928 (an increase of 2.1%), 0.913 (an increase of 0.4%), and 0.892 (a decrease of 1.7%) on the test set, respectively. The original model had an AUC of 0.909.

[0417] Improved device coverage: Through multi-level lightweight and dynamic resource-aware deployment, the system can cover 97% of terminal devices in the medical network, an increase of 66% compared to traditional deployment methods (which only cover 31% of high-performance devices).

[0418] Cross-regional adaptability improvement effect:

[0419] Through regional adaptive hierarchical learning, this method effectively solves the model migration difficulties caused by data distribution differences between different regions:

[0420] Stable performance within the region: Within their respective training areas, the model performance (F1 score) for the city center, suburban area, and remote suburbs was 0.87, 0.85, and 0.84, respectively, with a difference of no more than 3.5%.

[0421] Improved cross-regional generalization capabilities: When the city center model is directly applied to remote suburban areas, the F1 score increases from 0.61 of the traditional model to 0.79, and the performance degradation is reduced from the original 29.8% to 9.2%, an improvement of 69.1%.

[0422] Regional feature adaptability: The model can automatically identify and adapt to the differences in feature importance across regions. For example, the model in the city center is more sensitive to BMI and blood lipid indicators, while the model in the suburban area is more sensitive to blood sugar fluctuations and dietary records.

[0423] Prediction stability in boundary regions: For medical institutions located at the junction of regions, the model with hybrid adaptive layers improves the average F1 score by 6.8% compared to the single region model.

[0424] Continuous adaptation to new data: Through a dynamic update mechanism, the model can gradually adapt to changes in the disease spectrum within the region. During the six-month observation period, the performance of the models in each region on the newly added data steadily improved, with an average increase of 2.3%.

[0425] In summary, this application example fully verifies the effectiveness and superiority of the lightweight promotion method of chronic disease models based on big data provided by this application in actual medical scenarios, especially in the two core technical challenges of balancing lightweight and accuracy and cross-regional adaptability.

[0426] like Figures 2 to 5As shown, they are model complexity and prediction accuracy; performance indicators of model variants on different devices; regional adaptability improvement effect; and terminal device type distribution.

[0427] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Ordinary technicians in this field can also make more forms of equivalent embodiments based on the inspiration of this embodiment, all of which are protected by this embodiment.

Claims

1. A lightweight promotion method for chronic disease models based on big data, characterized by: The following steps are involved: Using multi-level federated distillation with differential privacy protection, medical institutions can collaborate on training basic models without sharing raw data. Based on the basic model, medical institutions are grouped according to their geographical location and disease spectrum characteristics. A regional adaptive layer is constructed for each group to achieve regional adaptability of the model and obtain a regional adaptive model. Based on the regional adaptability model, we perform resource profiling on terminal devices and classify them based on federated clustering. We generate model variants of corresponding complexity for different resource levels and ensure knowledge transfer between model variants through progressive knowledge distillation. Based on the generated model variants, the device resource status is monitored in real time, and the model variant version that best suits the current resource status is selected based on the scoring function to achieve dynamic deployment; Based on the deployed model variants, local feedback data from each medical institution is collected through a differentially privacy-protected model update mechanism to achieve continuous optimization of the model.

2. The lightweight promotion method of chronic disease model based on big data according to claim 1 is characterized in that: The steps of multi-level federated distillation using differential privacy protection include: Each medical institution uses local patient data to train the initial model; Each medical institution uses a public test dataset to generate statistical information about the model output and adds noise that satisfies differential privacy. The central server collects the differentially private output statistics generated by each medical institution and integrates this information using a weighted average method; The central server uses the integrated global output statistics as the output of the teacher model to train a student model with fewer parameters and lower computational complexity; The central server distributes the trained lightweight student model as the basic model to each participating institution.

3. The method for lightweight promotion of chronic disease models based on big data according to claim 1 is characterized in that: The step of grouping medical institutions according to geographical location and disease spectrum characteristics includes: All participating medical institutions were divided into multiple groups based on their geographical location, service population characteristics, disease spectrum distribution, and other information; For each group, the medical institutions within it use the distributed lightweight basic model as the initial model, fine-tune it based on their respective local data, and then use the federated averaging algorithm to generate a group-specific model; Knowledge exchange between groups is carried out through cross-regional knowledge distillation technology, so that the models of each group can learn from each other while maintaining their respective regional characteristics.

4. The method for lightweight promotion of chronic disease models based on big data according to claim 3 is characterized in that: The step of constructing a regional adaptation layer for each group includes: On the basis of the basic model, a lightweight regional adaptation layer is added to each group to form a complete regional adaptation model; The regional adaptation layer adopts at least one of a residual adaptation structure, a feature conversion structure, an attention modulation structure or a conditional normalization structure; The degree of regional specialization is controlled by the regional specialization coefficient, which is a fixed value or a dynamic parameter that is automatically adjusted according to the characteristics of the input data.

5. The method for lightweight promotion of chronic disease models based on big data according to claim 3 is characterized in that: The step of obtaining the regional adaptability model includes: The central server combines the basic model with the corresponding regional adaptive layer to generate a lightweight model for each institution that is suitable for its regional characteristics; For medical groups with multiple branches, a comprehensive model with multiple regional adaptive layers is generated; For institutions located at the boundary of regions, a hybrid model is generated that fuses specialized layers from multiple adjacent regions.

6. The method for lightweight promotion of chronic disease models based on big data according to claim 1 is characterized in that: The steps of performing resource profiling on the terminal devices and classifying them based on federated clustering include: Collect resource information of various terminal devices in each medical institution, including processor performance, memory capacity, storage space, power status and network connection; Integrate the resource information of all devices into a device resource profile; The federated K-means clustering algorithm is used to classify device resource profiles and divide devices into three resource levels: high, medium, and low.

7. The method for lightweight promotion of chronic disease models based on big data according to claim 1 is characterized in that: The step of generating model variants of corresponding complexity for different resource levels includes: For high-resource-level devices, the regional adaptive model is directly used as a high-complexity model variant; For medium-resource devices, network pruning and selective quantization techniques are used to generate medium-complexity model variants. For low-resource devices, model distillation and structural reorganization techniques are used to generate low-complexity model variants; Based on the hardware characteristics of the target device, each model variant is optimized for computational graphs, memory access, and dedicated instruction sets.

8. The method for lightweight promotion of chronic disease models based on big data according to claim 1 is characterized in that: The step of selecting the model variant version that best suits the current resource status based on the scoring function includes: Run a lightweight resource monitoring module on the device to collect device operating status information in real time; Based on the resource profile of the current device, select the most suitable version from multiple model variants; The scoring function comprehensively considers model performance and resource consumption, and balances accuracy and resource consumption through weight parameters; When resource status changes significantly, a smooth switching strategy is adopted to achieve seamless transition between model variants.

9. The method for lightweight promotion of chronic disease models based on big data according to claim 1 is characterized in that: The model update mechanism for differential privacy protection includes: Each medical institution collects local prediction feedback data to form a feedback data set; Use local feedback data to fine-tune the current model and generate an improved local model; Extract the knowledge increments of the local model improvement and add noise that satisfies differential privacy; The central server collects the knowledge increments submitted by each institution and aggregates them using a weighted average method to generate global model updates; The central server converts the global knowledge increment into model parameter updates and sends them to each organization.

10. A system for lightweight promotion of chronic disease models based on big data, used to implement a method for lightweight promotion of chronic disease models based on big data according to any one of claims 1 to 9, characterized in that: include: A multi-level federated distillation module that enables medical institutions to collaboratively train base models without sharing raw data; A regional adaptive hierarchical learning module is used to group medical institutions based on geographic location and disease spectrum characteristics, and to construct a regional adaptive layer for each group; The device resource profiling and model variant generation module is used to profile the terminal devices and classify them based on federated clustering, generating model variants of corresponding complexity for different resource levels; Dynamic resource perception module, which monitors the device resource status in real time and selects the model variant that best suits the current resource status based on a scoring function; The model update module is used to continuously optimize the model based on local feedback data from various medical institutions through a differential privacy protection mechanism.

Images