Dynamic risk assessment method based on multi-source heterogeneous data and application

Abstract

The invention discloses a dynamic risk assessment method based on multi-source heterogeneous data and application. The method comprises the following steps: extracting risk features from the multi-source heterogeneous data; constructing a multi-dimensional risk assessment model based on the labeled risk features; and based on the multi-dimensional risk assessment model, the risk is scored. According to the method, multi-source heterogeneous data can be effectively fused, risk features are extracted through a unified data processing flow, a multi-dimensional risk assessment model is constructed, risk scoring based on the multi-dimensional risk features is realized, the problems of coarse assessment granularity and insufficient information coverage caused by dependence on a single data source in the prior art are solved, and the risk assessment efficiency is improved. And the comprehensiveness and the accuracy of a risk assessment result are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of information processing technology, and more specifically, relates to a dynamic risk assessment method and application based on multi-source heterogeneous data. Background Technology

[0002] Currently, risk assessment methods face significant bottlenecks in terms of data integration capabilities, model adaptability, and real-time processing efficiency. For example, for chronic disease risk assessment, traditional solutions often rely on electronic medical record data from a single source, making it difficult to integrate multidimensional and heterogeneous data such as physical examination center results, wearable device monitoring information, family genetic history, and behavioral habits into a unified framework, resulting in coarse assessment granularity and insufficient coverage.

[0003] The existing invention patent with publication number CN117038065A proposes a method and system for predicting chronic disease risk by combining patient network analysis and graph learning. The method includes: S1, data preprocessing; S2, constructing a patient-disease bipartite graph based on the initial dataset and its basic features; S3, constructing patient networks based on local disease overlap, global disease overlap, and disease embedding based on the patient-disease bipartite graph; S4, constructing a chronic disease prediction model based on the patient networks based on local disease overlap, global disease overlap, and disease embedding from S3; and S5, predicting disease risk for any prediction target using the chronic disease prediction model based on its historical hospitalization information. This approach directly converts all hospitalization records into a patient-disease bipartite graph, resulting in a single data source, coarse assessment granularity, and insufficient coverage. Summary of the Invention

[0004] To overcome the problem of single data sources in existing risk assessment methods, this invention provides a dynamic risk assessment method and its application based on multi-source heterogeneous data.

[0005] The primary objective of this invention is to solve the aforementioned technical problems. The technical solution of this invention is as follows: The first aspect of this invention provides a dynamic risk assessment method based on multi-source heterogeneous data, comprising the following steps: Extracting risk features from multi-source heterogeneous data; Based on the labeled risk characteristics, a multidimensional risk assessment model is constructed; Risk is scored based on a multidimensional risk assessment model.

[0006] Furthermore, risk features are extracted from multi-source heterogeneous data, including time-driven mode and event-driven mode. The time-driven mode extracts multi-source heterogeneous data periodically in full or incremental form. The event-driven mode triggers the extraction of multi-source heterogeneous data immediately when an anomaly is detected.

[0007] Furthermore, risk features are extracted from multi-source heterogeneous data, including the following steps: Acquire multi-source heterogeneous data, perform intelligent rule base dynamic matching on the data, output candidate anomaly features and write them into the pending confirmation mark buffer pool; Set a sliding time window for each type of candidate anomaly feature. The window size is determined based on the feature change frequency, and the sliding step size is set to be smaller than the window width. The window slides across the time series with a fixed step size to perform statistical analysis and evolution calculations on the feature data and determine whether there is a specific trend of change. If a feature exhibits a clear abnormal trend in multiple consecutive sliding windows, the feature is marked as an abnormal feature. Within a preset confirmation period, abnormal features in the buffer pool are evaluated a second time using newly entered feature data. If abnormalities are still found in the second evaluation, the abnormal features are output; otherwise, the abnormal features are removed from the buffer pool.

[0008] Furthermore, the multidimensional risk assessment model constructs a multidimensional tensor representation of the risk space using risk level, risk type, and risk factor type as dimensions, and sets a weight coefficient matrix under each dimension. The weight coefficient matrix is ​​obtained by fitting and training with real data, including the following steps: Collect multi-source heterogeneous historical data, clean and standardize the collected data, and divide it into training set and validation set; A multidimensional risk assessment model is constructed using the standardized training set, with a weight coefficient matrix for each dimension. The task is to initialize the machine learning model and construct the prediction weight coefficient matrix. The specific indicator data of each dimension of the multidimensional risk assessment model are input into the machine learning model, and the weight coefficient matrix is ​​output. The mean square loss function value is calculated based on the difference between the weight coefficient matrix and the actual weight coefficient matrix used. The model parameters are continuously adjusted using gradient descent to minimize the loss function value. In each iteration, the model parameters are updated according to the gradient direction of the loss function until the loss function converges to a preset threshold or reaches the maximum number of iterations, thus obtaining the trained model and the weight coefficient matrix. The trained model is evaluated using cross-validation on a validation set. If the validation error meets the preset accuracy threshold, the model training is complete, and a stable model and weight coefficient matrix are output. If the validation error is too large, the cause is analyzed and adjustments are made until the validation error meets the preset accuracy threshold.

[0009] Furthermore, the machine learning model is a multi-task learning model or a neural network model.

[0010] Furthermore, the weight coefficient matrix has a dynamic evolution mechanism, including the following steps: The system presets a data update cycle and automatically triggers a data collection task to incrementally extract data when the cycle is reached. The newly extracted data is cleaned, and missing values ​​are removed, logical consistency is checked, and outliers are identified by combining the preset verification rule base. The data subset that meets the standard is then output. The Z-score method is used to normalize the data subset, mapping index values ​​from different sources to a unified dimension, as shown in the following expression:

[0011] in, This is the standard value. The collected data values, This is the average of historical data. Standard deviation; The normalized data is merged with the original training dataset, and a version identifier is assigned to the updated training set. The weight coefficient matrix is ​​retrained using the updated training set. Backpropagation optimization is performed using the mean square error between the predicted output base score and the true base score as the loss function until the loss function converges, resulting in the updated weight coefficient matrix. The corresponding training dataset version is associated with the version identifier. By calculating the difference between the updated weight coefficient matrix and the current weight coefficient matrix through cosine similarity comparison, when the difference exceeds a set threshold, the updated weight coefficient matrix is ​​temporarily stored in the weight library to be activated. When the data update cycle is detected to have expired, or when the AUC index of the applied weight coefficient matrix is ​​detected to be lower than the preset threshold for m consecutive times, the applied weight coefficient matrix is ​​replaced with the weight coefficient matrix in the weight library to be activated, and the adjustment time, reason and adjustment range of the weight coefficient matrix are recorded using the version control system. When an anomaly occurs, the weight coefficient matrix is ​​rolled back to the historical version with one click.

[0012] Furthermore, the risk is scored, including a base score and a synergy score, comprising the following steps: Obtain new input data for the current risk assessment, and perform standardized processing on the data to obtain standardized indicator values; The base score is calculated using the weight coefficient matrix obtained during training and the standardized index values. The expression is as follows: Where n is the total number of features. Let i be the standardized index value of the i-th feature factor. is the weight of the i-th feature factor obtained from the weight coefficient matrix; If a high-risk combined feature is identified from the standardized index values, a non-linear fitting model is established using real data to determine the synergy enhancement coefficient, and the synergy score is calculated by combining the basic score; if no high-risk combined feature is identified, the synergy score is 0, and the synergy score has the following expression:

[0013] where is a high-risk combination, is the sum of the basic scores of each single risk factor in the high-risk combination, is the corresponding synergy enhancement coefficient of this combination in the synergy enhancement coefficient matrix; The basic score and the synergy score are synthesized to obtain the total risk score.

[0014] Furthermore, determining the synergy enhancement coefficient using non-linear fitting modeling with real data includes the following steps: Construct a preset high-risk combination template based on the risk co-occurrence rule; Use the association rule mining algorithm to discover new risk combinations from historical feature data and add them to the high-risk combination template. For each type of combination in the high-risk combination template, calculate the risk gain value through the following expression :

[0015] where C1 and C2 respectively represent two basic risks, and P(·) represents the risk occurrence probability under this combination condition, represents the probability that risks C2 and C1 occur simultaneously; Encode the combined features of the historical feature data after standardization processing and output the structured combined feature identifier; Assign risk gain values to each combined feature identifier according to the high-risk combination template, and map the risk gain values to the standardized interval using the maximum-minimum normalization method to obtain the combined risk gain vector set; Input the combined risk gain vector set into the random forest regression model, and use Bayesian search optimization to output the preliminary synergy enhancement coefficient matrix; Analyze the contribution degree of each combined feature to the synergy enhancement coefficient according to the SHAP value, remove redundant combinations with a contribution degree < k, trigger the expert review process for abnormal items with an absolute value of the coefficient greater than m, until all abnormal items are eliminated, output the final synergy enhancement coefficient matrix, and extract the synergy enhancement coefficient from the final synergy enhancement coefficient matrix.

[0016] Furthermore, using the A / B evaluation mechanism to continuously verify the accuracy of the method prediction and adjust the weight coefficient matrix and the synergy enhancement coefficient includes the following steps: A preset evaluation and update cycle is established. When the cycle is reached, the updated weight coefficient matrix and co-enhancement coefficients are obtained by retraining using the new data. The newly generated user data to be evaluated is randomly divided into a test group and a control group according to a set ratio. The test group data is input into a scoring method based on the new weight coefficient matrix and the synergy enhancement coefficient, and the control group data is input into a scoring method based on the current weight coefficient matrix and the synergy enhancement coefficient. The unique user identifier ensures that the same user data is always fixed in the same group. Collect the scoring results of the two sets of models respectively, calculate the difference between the model scoring results and the actual scoring results, and statistically analyze one or more of the following performance indicators: AUC value, ROC curve, accuracy, recall, and additionally calculate the F1 value for high-risk combinations involving collusion scores. If the performance indicators of the test group are improved by more than the preset threshold compared with the control group and meet the preset number of consecutive days, the test group is determined to be better and the corresponding weight coefficient matrix and synergy enhancement coefficient are updated to the effective version. The weight coefficient matrix, synergy enhancement coefficient and performance indicators before and after the update are recorded and saved through logs to support traceability and rollback. If the test group model fails to pass the screening conditions, the existing weight coefficient matrix and synergy enhancement coefficient are retained as the effective version.

[0017] The second aspect of this invention provides an application of a dynamic risk assessment method based on multi-source heterogeneous data, which is applied to the screening of multi-source heterogeneous chronic diseases. The multi-source heterogeneous data includes one or more of the following: physical examination data, electronic medical records, wearable device data, behavioral data, and genetic characteristics.

[0018] Furthermore, the multi-source heterogeneous chronic disease screening data are dynamically matched using an intelligent rule base. The intelligent rule base includes physiological rule groups, biochemical rule groups, behavioral rule groups, and genetic rule groups. Each rule group is updated synchronously with mainstream clinical guidelines through a standard interface.

[0019] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention can effectively integrate multi-source heterogeneous data, extract risk features through a unified data processing flow, construct a multi-dimensional risk assessment model, and realize risk scoring based on multi-dimensional risk features. It overcomes the problems of coarse assessment granularity and insufficient information coverage caused by relying on a single data source in the prior art, and significantly improves the comprehensiveness and accuracy of risk assessment results. Attached Figure Description

[0020] To make the objectives and technical solutions of this invention clearer, the following drawings are provided and described: Figure 1 A flowchart illustrating a dynamic risk assessment method based on multi-source heterogeneous data provided in this embodiment of the invention; Figure 2 A flowchart for risk feature extraction provided in this embodiment of the invention; Figure 3 This is a flowchart of the weight coefficient matrix training method provided in an embodiment of the present invention; Figure 4 A flowchart of the scoring method provided in an embodiment of the present invention; Figure 5 A flowchart illustrating the method for calculating the synergistic enhancement coefficient provided in an embodiment of the present invention. Detailed Implementation

[0021] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0022] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0023] Example 1: This invention provides a dynamic risk assessment method based on multi-source heterogeneous data, such as... Figure 1 The diagram shows a flowchart of a dynamic risk assessment method based on multi-source heterogeneous data. The specific steps are as follows: S1: Extract risk features from multi-source heterogeneous data.

[0024] More specifically, risk features are extracted from multi-source heterogeneous data, including time-driven and event-driven modes. The time-driven mode involves periodically extracting full or incremental data from multi-source heterogeneous data; the event-driven mode triggers the extraction of multi-source heterogeneous data immediately upon detecting an anomaly, such as... Figure 2 As shown, the specific process is as follows: S1.1: Acquire multi-source heterogeneous data, perform intelligent rule base dynamic matching on the data, output candidate abnormal features and write them into the pending confirmation mark buffer pool; S1.2: Set a sliding time window for each type of candidate anomaly feature. The window size is determined based on the feature change frequency, and the sliding step size is set to be smaller than the window width. S1.3: The window slides across the time series with a fixed step size to perform statistical analysis and evolution calculations on the feature data and determine whether there is a specific trend of change; S1.4: If a feature shows a clear abnormal trend in multiple consecutive sliding windows, mark the feature as an abnormal feature; S1.5: Within the preset confirmation period, perform a secondary evaluation on the abnormal features in the buffer pool using the newly entered feature data. If the secondary evaluation still finds abnormalities, output the abnormal features; otherwise, remove the abnormal features from the buffer pool.

[0025] S2: Construct a multidimensional risk assessment model based on the labeled risk characteristics.

[0026] More specifically, the multidimensional risk assessment model constructs a multidimensional tensor representation of the risk space using risk level, risk type, and risk factor type as dimensions, and sets a weight coefficient matrix under each dimension. The weight coefficient matrix is ​​obtained by fitting and training with real data, such as... Figure 3 As shown, it includes the following steps: S2.1.1: Collect multi-source heterogeneous historical data, clean and standardize the collected data, and divide it into training set and validation set; S2.1.2: Construct a multidimensional risk assessment model using the standardized training set, with a weight coefficient matrix for each dimension; S2.1.3: Initialize the machine learning model and construct the prediction weight coefficient matrix; S2.1.4: Input the specific indicator data of each dimension of the multidimensional risk assessment model into the machine learning model, output the weight coefficient matrix, and calculate the mean square loss function value based on the difference between the weight coefficient matrix and the actual weight coefficient matrix used. S2.1.5: Use gradient descent to continuously adjust the model parameters to minimize the loss function value. In each iteration, update the model parameters according to the gradient direction of the loss function until the loss function converges to the preset threshold or reaches the maximum number of iterations, and obtain the trained model and weight coefficient matrix. S2.1.6: Use the validation set to evaluate the trained model through cross-validation. If the validation error meets the preset accuracy threshold, the model training is complete, and a stable model and weight coefficient matrix are output. If the validation error is too large, analyze the reasons and make adjustments until the validation error meets the preset accuracy threshold.

[0027] More specifically, the machine learning model is a multi-task learning model or a neural network model.

[0028] More specifically, the weighting coefficient matrix has a dynamic evolution mechanism, including the following steps: S2.2.1: Preset data update cycle, and automatically trigger the data acquisition task to incrementally extract data when the cycle is reached; S2.2.2: Clean the newly extracted data, and combine it with the preset verification rule base to remove missing values, verify logical consistency and identify outliers, and output a data subset that meets the standard; S2.2.3: The Z-score method is used to normalize the data subset, mapping index values ​​from different sources to a unified dimension. The expression is as follows:

[0029] in, This is the standard value. The collected data values, This is the average of historical data. Standard deviation; S2.2.4: Merge the normalized data with the original training dataset and assign a version identifier to the updated training set; S2.2.5: Retrain the weight coefficient matrix using the updated training set, and perform backpropagation optimization using the mean square error between the predicted output base score and the true base score as the loss function until the loss function converges, to obtain the updated weight coefficient matrix, and associate it with the corresponding training dataset version through the version identifier. S2.2.6: By calculating the difference between the updated weight coefficient matrix and the current weight coefficient matrix through cosine similarity comparison, when the difference exceeds the set threshold, the updated weight coefficient matrix is ​​temporarily stored in the weight library to be activated. S2.2.7: When it is detected that the data update cycle has expired, or when the AUC index of the weight coefficient matrix being applied is detected to be lower than the preset threshold for m consecutive times, the weight coefficient matrix being applied will be replaced with the weight coefficient matrix in the weight library to be effective, and the adjustment time, reason and adjustment range of the weight coefficient matrix will be recorded using the version control system. When an anomaly occurs, the weight coefficient matrix will be rolled back to the historical version with one click.

[0030] S3: Based on a multidimensional risk assessment model, risks are scored, such as... Figure 4 As shown, the specific process is as follows: S3.1: Obtain new input data to be assessed for risk, standardize the data, and obtain standardized index values.

[0031] S3.2: Calculate the base score using the weight coefficient matrix obtained during training and the standardized index values. The expression is as follows: Where n is the total number of features. Let i be the standardized index value of the i-th feature factor. The weight of the i-th feature factor is obtained from the weight coefficient matrix.

[0032] S3.3: If high-risk combined features are identified from the standardized index values, use non-linear fitting modeling of real data to determine the synergy enhancement coefficient, and calculate the synergy score by combining the basic score; if no high-risk combined features are identified, the synergy score is 0, and the expression of the synergy score is as follows:

[0033] where, is the high-risk combination, is the total basic score of each single risk factor in the high-risk combination, is the corresponding synergy enhancement coefficient of this combination in the synergy enhancement coefficient matrix.

[0034] More specifically, use non-linear fitting modeling of real data to determine the synergy enhancement coefficient. As Figure 5 shown, it includes the following steps: S3.3.1: Construct a preset high-risk combination template based on the risk co-occurrence rule; S3.3.2: Use the association rule mining algorithm to discover new risk combinations from historical feature data and add them to the high-risk combination template. For each type of combination in the high-risk combination template, calculate the risk gain value through the following expression :

[0035] where, C1 and C2 respectively represent two basic risks, P(·) represents the risk occurrence probability under this combination condition, represents the probability that risk C2 and C1 occur simultaneously; S3.3.3: Perform combined feature encoding on the standardized historical feature data and output the structured combined feature identifier; S3.3.4: Assign risk gain values to each combined feature identifier according to the high-risk combination template, and use the maximum-minimum normalization method to map the risk gain values to the standardized interval to obtain the combined risk gain vector set; S3.3.5: Input the combined risk gain vector set into the random forest regression model, optimize it using Bayesian search, and output the preliminary synergy enhancement coefficient matrix; S3.3.6: Analyze the contribution degree of each combined feature to the synergy enhancement coefficient according to the SHAP value, remove the redundant combinations with a contribution degree <k, trigger the expert review process for the abnormal items with an absolute value of the coefficient greater than m, until all abnormal items are eliminated, output the final synergy enhancement coefficient matrix, and extract the synergy enhancement coefficient from the final synergy enhancement coefficient matrix.

[0036] S3.4: Synthesize the basic score and the synergy score to obtain the total risk score.

[0037] More specifically, the accuracy of the method's predictions is continuously validated using an A / B evaluation mechanism, and the weighting coefficient matrix and synergistic enhancement coefficients are adjusted, including the following steps: A preset evaluation and update cycle is established. When the cycle is reached, the updated weight coefficient matrix and co-enhancement coefficients are obtained by retraining using the new data. The newly generated user data to be evaluated is randomly divided into a test group and a control group according to a set ratio. The test group data is input into a scoring method based on the new weight coefficient matrix and the synergy enhancement coefficient, and the control group data is input into a scoring method based on the current weight coefficient matrix and the synergy enhancement coefficient. The unique user identifier ensures that the same user data is always fixed in the same group. Collect the scoring results of the two sets of models respectively, calculate the difference between the model scoring results and the actual scoring results, and statistically analyze one or more of the following performance indicators: AUC value, ROC curve, accuracy, recall, and additionally calculate the F1 value for high-risk combinations involving collusion scores. If the performance indicators of the test group are improved by more than the preset threshold compared with the control group and meet the preset number of consecutive days, the test group is determined to be better and the corresponding weight coefficient matrix and synergy enhancement coefficient are updated to the effective version. The weight coefficient matrix, synergy enhancement coefficient and performance indicators before and after the update are recorded and saved through logs to support traceability and rollback. If the test group model fails to pass the screening conditions, the existing weight coefficient matrix and synergy enhancement coefficient are retained as the effective version.

[0038] Example 2: This embodiment provides an application of a dynamic risk assessment method based on multi-source heterogeneous data, using the dynamic risk assessment method based on multi-source heterogeneous data described in Embodiment 1 to perform risk assessment on multi-source heterogeneous chronic disease screening data.

[0039] The specific steps are as follows: S1: Extract risk features from multi-source heterogeneous chronic disease screening feature data, perform hierarchical classification and labeling of the extracted feature data, and generate three types of labels based on the number of features, severity, and duration, according to the following logic: P0 (abnormal diagnosis): Meets ≥3 high-risk indicators; P1 (Suspected Abnormality): 2 abnormalities + 1 high-risk behavior; P2 (Requires Observation): One indicator remains abnormal for ≥30 days; The specific process is as follows: S1.1: Acquire multi-source heterogeneous data, perform intelligent rule base dynamic matching on the data, output candidate abnormal features contained in the risk feature labels of P0 and P1, and write them into the pending confirmation mark buffer pool.

[0040] More specifically, the acquisition of multi-source heterogeneous data includes a time-driven mode and an event-driven mode. The time-driven mode involves periodically (e.g., daily) extracting full or periodically incremental multi-source heterogeneous chronic disease screening feature data. The event-driven mode triggers the extraction of multi-source heterogeneous chronic disease screening feature data immediately when an anomaly is detected (e.g., when a rapid increase in blood pressure, abnormal fluctuations in postprandial blood glucose, or abnormal user behavior (e.g., prolonged lack of exercise)).

[0041] The intelligent rule base includes physiological rule groups, biochemical rule groups, behavioral rule groups, and genetic rule groups. Each rule group can be updated synchronously with mainstream clinical guidelines through a standard interface, or it can be customized by the hospital to achieve flexible adjustment of feature indicators, thresholds, and combination logic.

[0042] Create a "Pending Confirmation Tag Buffer Pool" to temporarily store feature annotation results that are suspected anomalies but have not yet been definitively confirmed. The data in the buffer pool is arranged in order of entry time, and a maximum storage time is set, such as 24 hours. Data exceeding this time will be automatically removed.

[0043] When the system initially determines that a feature meets the suspected anomaly conditions (such as satisfying the conditions of the P1 or P2 labels) based on the feature annotation logic, it stores the feature and its annotation result in the buffer pool. For example, if a user is detected to have two abnormal indicators and one high-risk behavior, and is initially determined to be P1 (suspected anomaly), the relevant feature data and the annotation are stored in the buffer pool.

[0044] S1.2: Set a sliding time window for each type of candidate anomaly feature. The window size is determined based on the feature change frequency, and the sliding step size is set to be smaller than the window width.

[0045] Define a time window for each feature data sequence. The window size is determined based on the frequency of feature changes and clinical experience. For example, for blood pressure data, which changes relatively quickly, the window size can be set to a shorter time interval, such as 1 hour; while for some genetic traits that change more slowly, the window size can be set to a longer time, such as 1 month.

[0046] S1.3: The window slides across the time series with a fixed step size to perform statistical analysis and evolution calculations on the feature data and determine whether there is a specific trend of change.

[0047] The window slides across the time series with a fixed step size. The step size should be chosen to balance computational efficiency and sensitivity to feature changes; generally, the step size should be smaller than the window size. For example, for the blood pressure data in the 1-hour window mentioned above, the step size could be set to 15 minutes. Each time the window slides, newly entered data is included in the calculation, while data outside the window is removed.

[0048] S1.4: If a feature shows a clear abnormal trend in multiple consecutive sliding windows, mark the feature as an abnormal feature.

[0049] Within each window, statistical analysis and evolution calculations are performed on the feature data. This includes calculating statistics such as the mean, variance, and rate of change of the feature, as well as determining whether the feature exhibits a specific trend, such as increasing, decreasing, or fluctuating. For example, by calculating the mean and rate of change of blood pressure data within a window, it can be determined whether blood pressure is stable or showing a rapid upward trend. If the mean blood pressure continues to rise and the rate of change exceeds a certain threshold across multiple consecutive windows, then blood pressure is considered to be showing an upward trend, and this feature is marked as an anomalous feature.

[0050] S1.5: Within the preset confirmation period, perform a secondary evaluation on the abnormal features in the buffer pool using the newly entered feature data. If the secondary evaluation still finds abnormalities, output the abnormal features; otherwise, remove the abnormal features from the buffer pool.

[0051] The specific process is as follows: During the buffer storage period, the system continuously monitors and re-evaluates the data in the buffer. This includes comprehensive analysis combining newly entered feature data and cross-validation using other relevant clinical data or models. For example, for suspected abnormal blood glucose data in the buffer, the system can further analyze the user's recent dietary records, exercise patterns, and other physiological indicators to help determine whether the abnormality is transient. If, during the buffer storage period, the re-evaluation confirms that the abnormality is a real and persistent problem, the abnormality feature is output; if the evaluation finds that the abnormality is caused by transient factors (such as measurement errors, temporary physiological fluctuations, etc.), the label is removed from the buffer and not included in subsequent risk assessments.

[0052] By combining the aforementioned feature evolution logic based on window sliding with the "buffer pool for confirmation" mechanism, the system can more accurately identify chronic disease risk characteristics, effectively avoid misjudgments caused by transient abnormalities, and improve the reliability of early screening risk assessment for chronic diseases. By accurately identifying high-risk groups (P0 / P1 labels) and reducing excessive testing for low-risk groups (P2 observation period), clinical data shows that unnecessary biochemical tests can be reduced by 25%-30%, while simultaneously increasing follow-up coverage of high-risk groups to over 90%, thus improving the efficiency of medical resource allocation.

[0053] S2: Construct a multidimensional risk assessment model based on the labeled risk characteristics.

[0054] More specifically, the multidimensional risk assessment model is a three-dimensional risk matrix with risk level labels, disease types, and risk factor types as dimensions, and a weight coefficient matrix is ​​set under each dimension.

[0055] More specifically, the three-dimensional risk matrix structure is defined as follows, with three dimensions including: Dimension 1: Risk Level (P0 Confirmed, P1 Suspected, P2 Under Observation); Dimension 2: Disease type (e.g., hypertension, diabetes and its complications). Dimension 3: Types of risk factors (physiological, biochemical, behavioral, genetic).

[0056] The weight coefficient matrix is ​​obtained by fitting and training with real data, including the following steps: S2.1: Collect multi-source heterogeneous historical data, clean and standardize (or normalize) the collected data, and divide it into training set and validation set.

[0057] S2.2: Construct a multidimensional risk assessment model using the standardized training set, with a weight coefficient matrix for each dimension.

[0058] S2.3: Initialize the machine learning model and construct the prediction weight coefficient matrix.

[0059] More specifically, select a machine learning model suitable for handling multi-dimensional, multi-classification problems, such as a multi-task learning model or a neural network model in deep learning. Taking a multi-task learning model as an example, the main task is to predict the weight coefficient matrix, while auxiliary tasks are set, such as predicting disease development trends or changes in risk levels. By sharing information among multiple tasks, the accuracy of the model's prediction of the weight coefficient matrix is ​​improved. Initialize the parameters of the machine learning model, including the learning rate, the number of iterations, and the number of neurons in the hidden layer.

[0060] S2.4: Input the specific indicator data of each dimension of the multidimensional risk assessment model into the machine learning model, output the weight coefficient matrix, and calculate the mean squared loss function value based on the difference between the weight coefficient matrix and the actual weight coefficient matrix used. The expression is as follows:

[0061] in, The actual weight coefficients of sample i. Here, n represents the weight coefficients output by the model for sample i, and n is the number of data samples.

[0062] S2.5: Use gradient descent to continuously adjust the model parameters to minimize the loss function value. In each iteration, update the model parameters according to the gradient direction of the loss function until the loss function converges to the preset threshold or reaches the maximum number of iterations, and obtain the trained model and weight coefficient matrix.

[0063] S2.6: Use the validation set to evaluate the trained model through cross-validation. If the validation error meets the preset accuracy threshold, the model training is complete, and a stable model and weight coefficient matrix are output. If the validation error is too large, analyze the reasons and make adjustments until the validation error meets the preset accuracy threshold.

[0064] On the other hand, the dynamic evolution mechanism of the weight coefficient matrix includes the following steps: The system has a preset data update cycle (e.g., 00:00 on the 1st of each month). Users can customize and adjust the cycle through the system management interface (supporting weekly, quarterly, and other granular modifications). When the cycle arrives, the system automatically triggers a data collection task. It incrementally extracts new real case data (including characteristic indicators of confirmed cases, diagnostic results, follow-up records, etc.) from multiple data sources such as the Hospital Information System (HIS), Laboratory Information Management System (LIS), and wearable device databases through standard interfaces. It adopts a scheduled task framework (e.g., based on SpringScheduler) combined with an incremental data extraction algorithm (based on timestamps or log offsets) to avoid the repeated transmission of full data and reduce system resource consumption.

[0065] The newly extracted data is cleaned and combined with the preset validation rule base to remove missing values, check logical consistency and identify outliers. It automatically identifies and removes data with missing key indicators (such as hypertension cases without blood pressure values), logical contradictions (such as age < 0) or obvious outliers (such as blood glucose > 30 mmol / L without clinical explanation), and outputs a data subset that meets the standards. The Z-score method is used to normalize the data subset, mapping index values ​​from different sources to a unified dimension, as shown in the following expression:

[0066] in, This is the standard value. The collected data values, This is the average of historical data. Standard deviation; The normalized data is merged with the original training dataset, and a version identifier (such as V202507) is assigned to the updated training set. Large-scale data is processed in parallel based on a distributed computing framework (such as Spark), and data cleaning is automatically performed through a preset verification rule base to ensure the consistency and compatibility of new data with historical data.

[0067] The updated training data is input into the machine learning model, and the weight coefficient matrix is ​​retrained using the gradient descent algorithm. The mean square error between the predicted output base risk score and the actual diagnostic base risk score is used as the loss function for backpropagation optimization until the loss function converges, resulting in the updated weight coefficient matrix. The corresponding training dataset version is associated with the version identifier (e.g., V202507). By calculating the difference between the updated weight coefficient matrix and the current weight coefficient matrix through cosine similarity comparison, when the difference exceeds a set threshold (in this embodiment, the threshold is set to 0.9), the updated weight coefficient matrix is ​​temporarily stored in the weight library to be activated. When it is detected that the data update cycle has expired, or when the AUC index of the applied weight coefficient matrix is ​​detected to be lower than the preset threshold (e.g., 0.85) for m consecutive times (in this embodiment, m is 3), the applied weight coefficient matrix is ​​replaced with the weight coefficient matrix in the weight library to be activated. The weight coefficient matrix adjustment trigger signal is transmitted through a message queue (e.g., Kafka), and the version control system (e.g., a lightweight implementation based on Git) records the adjustment time, reason, adjustment range, and other audit information of the weight coefficient matrix. When an anomaly occurs, the weight coefficient matrix can be rolled back to a historical version with one click.

[0068] S3: Based on a multidimensional risk assessment model, risk is scored.

[0069] More specifically, a multidimensional risk assessment model is used to score the risk, the scoring including a basic score and a synergistic score, comprising the following steps: S3.1: Obtain new input data to be assessed for risk, standardize the data, and obtain standardized index values.

[0070] S3.2: Calculate the base score using the weight coefficient matrix obtained during training and the standardized index values. The expression is as follows: Where n is the total number of features. Let i be the standardized index value of the i-th feature factor. The weight of the i-th feature factor is obtained from the weight coefficient matrix.

[0071] S3.3: If high-risk combination characteristics are identified from the standardized index values, the synergy enhancement coefficient is determined using nonlinear fitting modeling with real data, and the synergy score is calculated in conjunction with the base score; if no high-risk combination characteristics are identified, the synergy score is 0, and the synergy score is... The expression is as follows:

[0072] in, This is a high-risk combination. It is the sum of the basic scores of each individual risk factor in the high-risk combination. This represents the synergistic enhancement coefficient corresponding to this combination in the synergistic enhancement coefficient matrix.

[0073] More specifically, the synergistic enhancement coefficients are determined using nonlinear fitting modeling with real data, including the following steps: S3.3.1: Construct 12 pre-defined high-risk combination templates based on comorbidity patterns (such as "hypertension + diabetes", "diabetes + hyperlipidemia", "obesity + hyperuricemia + family history", etc.). S3.3.2: Using an association rule mining algorithm (an improved version of Apriori), new combinations are discovered from historical data (minimum support set at 5%, confidence ≥ 80%) and added to the high-risk combination template. For each type of combination in the high-risk combination template, the risk gain value is calculated using the following expression. :

[0074] Where C1 and C2 represent two underlying chronic diseases or complications, respectively, and P(·) represents the probability of complication under this combination of conditions. This represents the probability that C2 and C1 occur simultaneously. S3.3.3: The standardized historical feature data is combined and coded (e.g., "hypertension = 1 + diabetes = 1" is marked as combination ID: 1001), and a structured combination feature identifier is output. Each combination feature identifier includes a combination feature code and a moderating variable. The moderating variable includes age stratification (<45 years / 45-65 years / >65 years), disease duration (<5 years / ≥5 years), and treatment adherence (high / medium / low); S3.3.4: A risk gain value is assigned to each combination feature identifier according to the high-risk combination template, and the risk gain value is mapped to a standardized interval using the max-min standardization method. In this embodiment, the standardized interval is the [0,2] interval (1 is no synergistic effect, >1 is risk enhancement, <1 is risk offset), resulting in a combination risk gain vector set; S3.3.5: Input the combined risk-gain vector set into the random forest regression model, and perform 5-fold stratified sampling (divided according to the proportion of combination types) on parameters such as the number of trees (using Bayesian optimization to search for the optimal value (range 50-500, optimal value 200 trees)), maximum depth (limited to 10 layers (to avoid overfitting)), and splitting feature selection (Gini coefficient). Ensure cross-validation that the sample distribution of each compromise combination is consistent, and output the preliminary co-enhancement coefficient matrix. The coefficient matrix structure is an N * N symmetric matrix (N = total number of combination types), and the matrix elements are... Let the synergistic enhancement coefficient of the i-th class and the j-th class combination satisfy the following condition: = ; S3.3.6: Analyze the contribution of each combination feature to the synergistic enhancement coefficient based on the SHAP value, remove redundant combinations with a contribution of <0.05, trigger the clinical expert review process for abnormal items with an absolute value of coefficient greater than m (in this embodiment, m is 3), until all abnormal items are eliminated, output the final synergistic enhancement coefficient matrix, and extract the synergistic enhancement coefficient from the final synergistic enhancement coefficient matrix.

[0075] S3.4: Combine the base score and the collaborative score to obtain the total risk score.

[0076] More specifically, the accuracy of the method's predictions is continuously validated using an A / B evaluation mechanism, and the weighting coefficient matrix and synergistic enhancement coefficients are adjusted, including the following steps: A preset evaluation and update cycle is established. When the cycle is reached, the updated weight coefficient matrix and co-enhancement coefficients are obtained by retraining using the new data. Newly generated user data to be evaluated is randomly divided into a test group and a control group according to a set ratio (user IDs are sharded using a hash algorithm to ensure that data from the same user always enters the same group). A custom allocation ratio (e.g., 2:8) can also be defined. Users can input allocation parameters in the configuration interface, and the ratio's validity is automatically verified (the sum is 100%). Grouping is performed based on a modulo operation using the user's unique identifier (e.g., a hash value of their ID card number), ensuring randomness and stability. A data isolation mechanism ensures that the two groups are calculated independently. Test group data is input into a scoring method based on a new weight coefficient matrix and synergistic enhancement coefficients, while control group data is input into a scoring method based on the current weight coefficient matrix and synergistic enhancement coefficients. The unique user identifier ensures that data from the same user always enters the same group. The scoring results of the two models were collected separately, and the difference between the model scores and the actual scores was calculated. One or more of the following performance metrics were statistically analyzed: AUC (Area Under Curve) value (assessing overall discrimination ability), ROC curve (showing sensitivity and specificity at different thresholds), precision (number of correct predictions / total number of predictions), and recall (the proportion of actual cases predicted). For high-risk combinations involving synergistic scores, an additional F1 score was calculated (balancing precision and recall). The formula is as follows:

[0077] Call statistical analysis engines (such as those based on the PythonScikit-learn library) to calculate indicators in batches and generate visual comparison reports (including AUC values, ROC curve overlays, and indicator trend line charts).

[0078] If the performance indicators of the test group improve by more than the preset threshold compared to the control group and meet the preset number of consecutive days (e.g., the AUC value improves by ≥5% compared to the control group for 3 consecutive days), the test group is determined to be superior, and the corresponding weight coefficient matrix and synergistic enhancement coefficient are updated to the effective version. The weight coefficient matrix, synergistic enhancement coefficient, and performance indicators before and after the update are recorded and saved through logs to support traceability and rollback. If the test group model fails to pass the screening conditions, the existing weight coefficient matrix and synergistic enhancement coefficient are retained as the effective version.

[0079] It can also perform screening and optimization judgments based on rule engines (such as Drools), automate the expert review process through workflow engines, and save model iteration logs (including indicator changes and effective time for each update) to support traceability analysis.

[0080] S4: Map the total risk score to n warning levels to obtain the risk assessment results.

[0081] More specifically, the total risk score is mapped to a three-color warning level (red / yellow / green), and corresponding health management suggestions are automatically generated. Furthermore, user behavior data is automatically fed back to the model input, and combined with subsequent follow-up data, it assists in model optimization, forming a complete closed loop of "assessment-intervention-feedback-optimization," which improves the model's ability to track changes in individual risk. Feature extraction and risk calculation support a lightweight edge deployment mode (such as ARM devices), which is suitable for primary healthcare scenarios, reduces network and computing power dependence, and solves the problems of "high deployment threshold and large response latency" of traditional systems in primary healthcare scenarios, thus improving the universality of the technology.

[0082] Table 1 shows the experimental results of a comparative experiment using the present invention and the traditional weighted method, selecting three typical high-risk combination scenarios. The experimental data came from a provincial chronic disease management database (January 2023 - June 2024, containing 123,894 valid samples). The false positive rate in the table represents the proportion of cases predicted as high-risk by the model but without complications within one year of follow-up; the false negative rate represents the proportion of cases predicted as low-risk by the model but with complications within one year. Statistical methods used included paired chi-square test; P < 0.001 was considered statistically significant.

[0083] Table 1

[0084] The experimental results show that this invention solves the technical pain point of difficulty in quantifying the synergistic effect of multiple diseases in traditional scoring models by using nonlinear models such as random forest regression specifically for training synergistic enhancement coefficients and achieving accurate generation of weight coefficient matrices through hierarchical feature engineering and clinical validation mechanisms. This invention can significantly reduce the risk of false positives / false negatives in combined disease scenarios.

[0085] Through the aforementioned dynamic evolution mechanism, the system can continuously absorb new data, optimize model parameters, and ensure a steady improvement in the model's predictive accuracy through scientific evaluation methods. This solves the problem of insufficient adaptability of traditional risk assessment models due to outdated data or fixed parameters, and achieves an upgrade from a "static model" to an "adaptive evolution model".

[0086] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A dynamic risk assessment method based on multi-source heterogeneous data, characterized in that, The method comprises the following steps: extracting risk features from multi-source heterogeneous data; constructing a multi-dimensional risk assessment model based on the labeled risk features; scoring the risk based on the multi-dimensional risk assessment model. 2.The dynamic risk assessment method based on multi-source heterogeneous data according to claim 1, characterized in that, The risk features are extracted from multi-source heterogeneous data, including a time-driven mode and an event-driven mode. The time-driven mode is to periodically extract all or incrementally extract multi-source heterogeneous data. The event-driven mode is to immediately trigger the extraction of multi-source heterogeneous data when an anomaly is detected. 3.The dynamic risk assessment method based on multi-source heterogeneous data according to claim 1, characterized in that, The risk features are extracted from multi-source heterogeneous data, including the following steps: Obtain multi-source heterogeneous data, and perform intelligent rule library dynamic matching on the data to output candidate abnormal features and write them into a to-be-confirmed label buffer pool; Set a sliding time window for each type of candidate abnormal feature, and determine the window size according to the feature change frequency, and set the sliding step size to be less than the window width; The window slides on the time series at a fixed step size, and the feature data is statistically analyzed and evolutionally calculated to determine whether there is a specific change trend; If the feature is observed to have a significant abnormal trend in a plurality of consecutive sliding windows, the feature is marked as an abnormal feature; In a preset confirmation period, the abnormal features in the buffer pool are re-evaluated using newly entered feature data. If the re-evaluation still has an abnormality, the abnormal feature is output. If there is no abnormality, the abnormal feature is removed from the buffer pool. 4.The dynamic risk assessment method based on multi-source heterogeneous data according to claim 1, characterized in that, The multi-dimensional risk assessment model constructs a multi-dimensional tensor representing the risk space with risk levels, risk types, and risk factor types as dimensions, and sets a weight coefficient matrix under each dimension. The weight coefficient matrix is obtained by fitting and training with real data, including the following steps: Collect multi-source heterogeneous historical data, clean and standardize the collected data, and divide them into a training set and a validation set; Construct a multi-dimensional risk assessment model using the standardized training set, and set a weight coefficient matrix under each dimension; Initialize the machine learning model to build a prediction weight coefficient matrix task; Input the specific index data of each dimension of the multi-dimensional risk assessment model into the machine learning model to output the weight coefficient matrix. According to the difference between the weight coefficient matrix and the actual weight coefficient matrix, the mean square loss function value is calculated; Use the gradient descent method to continuously adjust the model parameters to minimize the loss function value. In each iteration process, update the model parameters according to the gradient direction of the loss function until the loss function converges to a preset threshold or reaches the maximum number of iterations to obtain a trained model and a weight coefficient matrix; Evaluate the trained model using the validation set through cross-validation. If the validation error meets the preset accuracy threshold, the model training is complete, and a stable model and a weight coefficient matrix are output. If the validation error is large, analyze the reason and adjust it until the validation error meets the preset accuracy threshold.

5. The method of claim 4, wherein, The machine learning model is a multi-task learning model or a neural network model.

6. The method of claim 4, wherein, The weight coefficient matrix has a dynamic evolution mechanism, including the following steps: Pre-set a data update period, and automatically trigger the data collection task to incrementally extract data when the period arrives; The extracted new data is cleaned, and missing values are removed, logical consistency is verified, and abnormal values are identified according to a preset verification rule library, and a data subset meeting the standard is output; The data subset is normalized by using the Z-score method, and the index values from different sources are mapped to a unified dimension, and the expression is as follows: wherein, is a standard value, is a collected data value, is a historical data mean value, is a standard deviation; The normalized data is combined with the original training data, and a version identifier is assigned to the updated training set; The updated weight coefficient matrix is retrained using the updated training set, and the mean square error of the predicted output base score and the real base score is used as the loss function for back propagation optimization until the loss function converges, obtaining the updated weight coefficient matrix, and the corresponding training data set version is associated through the version identifier; The difference between the updated weight coefficient matrix and the current weight coefficient matrix is compared by calculating the cosine similarity, and when the difference exceeds the set threshold, the updated weight coefficient matrix is temporarily stored in the to-be-effective weight library; When it is detected that the data update period has arrived, or the AUC index of the weight coefficient matrix being applied is lower than the preset threshold for m consecutive times, the weight coefficient matrix being applied is replaced with the weight coefficient matrix in the to-be-effective weight library, and the adjustment time, reason, and adjustment amplitude of the weight coefficient matrix are recorded using the version control system. When an exception occurs, the weight coefficient matrix is rolled back to the historical version.

7. The method of claim 4, wherein, The risk is scored, and the score includes a base score and a synergy score, including the following steps: Obtain new input data for risk assessment, standardize the data, and obtain standardized index values; Using the trained weight coefficient matrix, the base score is calculated in combination with the standardized index value , and the expression is as follows: wherein n is the total number of characteristics, is the standardized index value of the i-th characteristic factor, is the i-th characteristic factor weight obtained from the weight coefficient matrix. If high-risk combination features are identified from the standardized index values, use real data nonlinear fitting modeling to determine the synergistic enhancement coefficient, combine the basic score to calculate the synergistic score; if no high-risk combination features are identified, the synergistic score is 0, and the synergistic score The expression of the synergistic score is as follows: wherein, is a high-risk combination, is a sum of the basic scores of each single risk factor in the high-risk combination, is the corresponding synergistic enhancement coefficient of the combination in the synergistic enhancement coefficient matrix; Combine the base score and the synergy score to obtain the total risk score. 8.The dynamic risk assessment method based on multi-source heterogeneous data according to claim 7, characterized in that, Determine the synergy enhancement coefficient by nonlinear fitting modeling using real data, including the following steps: Based on the risk co-occurrence rule, a preset high-risk combination template is constructed; The new risk combination is discovered from the historical characteristic data by using the association rule mining algorithm and is added to the high-risk combination template. For each type of combination in the high-risk combination template, a risk gain value is calculated by the following expression : wherein C1 and C2 represent two basic risks, respectively, P(·) represents the probability of the occurrence of the risk under the combination condition, represents the probability of the occurrence of the risk C2 simultaneously with C1. The standardized historical feature data is combined and encoded, and a structured combination feature identifier is output; For each combination feature identifier, a risk gain value is assigned according to the high-risk combination template, and the risk gain value is mapped to a standardized interval using the max-min normalization method to obtain a combination risk gain vector set; The combination risk gain vector set is input into a random forest regression model, and a Bayesian search optimization is used to output a preliminary synergy enhancement coefficient matrix; According to the SHAP value analysis, the contribution of each combination feature to the synergy enhancement coefficient is analyzed, redundant combinations with a contribution degree < k are removed, and expert review procedures are triggered for abnormal items with a coefficient absolute value greater than m until all abnormal items are eliminated, and the final synergy enhancement coefficient matrix is output. The synergy enhancement coefficient is extracted from the final synergy enhancement coefficient matrix.

9. The dynamic risk assessment method based on multi-source heterogeneous data according to claim 8, characterized in that, The accuracy of the prediction is continuously verified using the A / B evaluation mechanism, and the weight coefficient matrix and the synergy enhancement coefficient are adjusted, including the following steps: A preset evaluation update period is set, and when the period arrives, the updated weight coefficient matrix and synergy enhancement coefficient are obtained by retraining using the new data; The newly generated user data to be evaluated is randomly divided into a test group and a control group according to a set proportion, the test group data is input into a scoring method based on a new weight coefficient matrix and a synergistic enhancement coefficient, the control group data is input into a scoring method based on a current weight coefficient matrix and a synergistic enhancement coefficient, and the same user data is always fixed into the same group through a unique user identifier; The scoring results of the two groups of models are collected respectively, the difference between the model scoring results and the true scoring results is calculated, and one or more of the following performance indicators is statistically analyzed: AUC value, ROC curve, accuracy, recall rate, and F1 value is additionally calculated for high-risk combinations involving synergistic scores; If the performance indicators of the test group improve more than the preset threshold compared with the control group and continuously meet the preset number of days, it is determined that the test group is better, and the corresponding weight coefficient matrix and synergistic enhancement coefficient are updated as the effective version, the weight coefficient matrix, synergistic enhancement coefficient and performance indicators before and after the update are recorded and saved through the log to support traceability and rollback; if the test group model fails to pass the optimization condition, the existing weight coefficient matrix and synergistic enhancement coefficient are continued to be retained as the effective version.

10. The application of the dynamic risk assessment method based on multi-source heterogeneous data according to any one of claims 1-9, characterized in that, It is applied to multi-source heterogeneous chronic disease screening, and the multi-source heterogeneous data includes one or more of the following: physical examination data, electronic medical records, wearable device data, behavior data, and genetic characteristics.

Images