A machine learning based data analysis method and system

Abstract

The application provides a machine learning-based data analysis method and system, belonging to the technical field of data processing and analysis. The method comprises: acquiring and fusing multiple source heterogeneous data sources of a target hospital; processing the multiple source heterogeneous data to obtain a target data set; storing the target data set to multiple storage partitions based on a preset data model and partition rule to obtain multiple data subsets; analyzing filter conditions in a user query request to match a target data subset from the multiple data subsets; selecting an analysis model from a preset model library according to an analysis target parameter in the query request and a partition attribute of the target data subset, analyzing the target data subset and outputting an analysis result; and generating a corresponding report based on the analysis result and interacting with the user through a graphical user interface. The application can improve the management level of the hospital.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing and analysis, and particularly relates to a data analysis method and system based on machine learning. BACKGROUND

[0002] In the process of modern hospital informatization construction, data management faces multiple challenges. Among them, the data of the hospital is scattered in HIS (hospital information system), ERP (enterprise resource planning system) and other multiple systems, forming a "data island", which not only hinders cross-department collaboration, but also leads to the fact that the accounting department cannot obtain patient billing information in time, affecting the accuracy and timeliness of data reports. Moreover, with the explosive growth of hospital data, the storage method is difficult to meet the demand of rapid query and complex analysis, and the daily maintenance of the database is also very difficult.

[0003] Therefore, there is an urgent need for a data analysis method and system based on machine learning to improve the management level of the hospital. SUMMARY

[0004] In order to solve the above technical problems, the present application provides a data analysis method and system based on machine learning.

[0005] The first aspect of the embodiment of the present application provides a data analysis method based on machine learning, comprising:

[0006] Obtaining and fusing processing of multiple source heterogeneous data sources of a target hospital, the multiple source heterogeneous data comprising: medical business system data, medical insurance settlement data and consumable supply chain data;

[0007] Processing the multiple source heterogeneous data to obtain a target data set;

[0008] Based on a preset data model and a partition rule, the target data set is stored in a plurality of storage partitions to obtain a plurality of data subsets; the partition rule supports at least one dimension of time range, business department and data type to divide data, and the partition granularity is adjusted according to the historical query frequency;

[0009] Analyzing the filtering conditions in the user query request, and matching the target data subset from the plurality of data subsets;

[0010] According to the analysis target parameters in the query request and the partition attributes of the target data subset, an analysis model is selected from a preset model library, the target data subset is analyzed and an analysis result is output;

[0011] Based on the analysis result, a corresponding report is generated, and the report is interacted with the user through a graphical user interface.

[0012] Optionally, the multiple source heterogeneous data of the target hospital comprises:

[0013] Obtain structured data and unstructured data of internal systems and external data platforms of the target hospital through an API interface;

[0014] Perform entity recognition and relationship extraction on the unstructured data using a natural language processing model to generate structured processing data;

[0015] Align the structured data and the structured processing data in terms of data patterns and unify the formats of the data;

[0016] Based on the weight coefficients of each data source of the target hospital, the aligned data is weighted and fused to generate multi-source heterogeneous data.

[0017] The present application can reasonably fuse data from different data sources according to the importance and reliability of the data, and improve the quality of the fused data, based on the weighted fusion of data source weight coefficients.

[0018] Optionally, the method for determining the weight coefficients of each data source comprises:

[0019] Obtain evaluation index data of each data source of the target hospital, including data update frequency index, historical accuracy rate index, and data provider reputation level index;

[0020] Calculate the weight coefficients of each data source according to the evaluation index data and a preset first weight calculation formula.

[0021] The present application can objectively and comprehensively reflect the importance and reliability of each data source by calculating the weight coefficients according to the evaluation index data such as data update frequency, historical accuracy rate, and data provider reputation level, thereby providing a scientific basis for weighted fusion of data.

[0022] Optionally, after calculating the weight coefficients of each data source according to the evaluation index data and a preset first weight calculation formula, the method further comprises:

[0023] In response to the data update frequency index being less than a first update frequency threshold, reduce the weight coefficient of the data source by a first step size;

[0024] In response to the data update frequency index being less than a second update frequency threshold and greater than or equal to the first update frequency threshold, reduce the weight coefficient of the data source by a second step size; wherein the first update frequency threshold is less than the second update frequency threshold, and the first step size is greater than the second step size.

[0025] The application dynamically adjusts the weight coefficient of the data source according to the comparison result of the data update frequency index and different threshold values, so that the weight coefficient can be optimized according to the real-time situation of the data, and the accuracy and reliability of data fusion are further improved.

[0026] Optionally, the target data set is stored into a plurality of storage partitions based on the preset data model and partition rules, including:

[0027] The target data set is converted into a structured data set according to a preset data model;

[0028] The time partition granularity is determined based on the historical query frequency, and the structured data set is divided into a primary storage partition according to a time range dimension;

[0029] The data records in the primary storage partition are divided into secondary sub-partitions according to a business department dimension;

[0030] The data records in the secondary sub-partitions are divided into tertiary sub-partitions according to a data type dimension;

[0031] A partition metadata index table is established to record the time range, business department code and data type code of each partition;

[0032] The data subsets after the tertiary division are distributed and stored into corresponding physical storage nodes to generate the plurality of data subsets.

[0033] The application converts the target data set into a structured data set and performs tertiary partition division, establishes a partition metadata index table, realizes structured storage and fine management of data, facilitates quick positioning and query of data, and improves the efficiency of data access.

[0034] Optionally, the time partition granularity is determined based on the historical query frequency, including:

[0035] The query frequency of different time intervals in the historical query request is counted;

[0036] According to the mapping relationship between the query frequency and the time interval, a daily, monthly or annual partition granularity is selected.

[0037] Optionally, according to the mapping relationship between the query frequency and the time interval, a daily, monthly or annual partition granularity is selected, including:

[0038] When the time interval meets the first time interval condition and the corresponding query frequency is greater than the first query frequency threshold, a daily partition granularity is adopted;

[0039] When the time interval meets the second time interval condition and the corresponding query frequency is greater than the second query frequency threshold, a monthly partition granularity is adopted;

[0040] When the time interval meets a third time interval condition and the corresponding query frequency is greater than a third query frequency threshold, a grade partition granularity is adopted.

[0041] The application selects a day-level, month-level or grade-level partition granularity by specifying different time interval conditions and query frequency thresholds, so that the selection of the partition granularity is more accurate and scientific, and the efficiency of data storage and query is further improved.

[0042] Optionally, the filtering condition in the user query request is analyzed, and a target data subset is matched from the plurality of data subsets, including:

[0043] The filtering condition in the user query request is analyzed to obtain a query time interval, a query department set and a query data type code;

[0044] The partition metadata index table is queried to filter partitions meeting the filtering condition to obtain filtered partitions;

[0045] The filtering condition includes that the partition time range overlaps the query time interval, the partition business department code belongs to the query department set, and the partition data type code matches the query data type code;

[0046] According to the overlap degree of the time range of the filtered partition and the query time interval, the loaded data is determined, and the loaded data is merged into the target data subset.

[0047] The application analyzes the filtering condition of the user query request, filters the partitions meeting the condition by querying the partition metadata index table, and determines the loaded data according to the overlap degree, which can accurately match the target data subset from the plurality of data subsets and reduce the data loading amount and query time.

[0048] Optionally, the determination of the loaded data according to the overlap degree of the time range of the filtered partition and the query time interval includes:

[0049] When the overlap degree is less than a first overlap threshold, the partition is skipped;

[0050] When the overlap degree is greater than or equal to the first overlap threshold and less than a second overlap threshold, the data overlapping the query time range and the partition time range is loaded;

[0051] When the overlap degree is greater than or equal to the second overlap threshold, the data of the partition is loaded.

[0052] The application formulates different data loading strategies according to the overlap degree of the filtered partition and the query time interval, which can load data according to the actual situation, avoid loading unnecessary data, and improve the data processing efficiency and resource utilization.

[0053] In a second aspect, the embodiment of the present application provides a data analysis system based on machine learning, comprising:

[0054] a data acquisition module configured to acquire and fuse a plurality of heterogeneous data sources of a target hospital, wherein the plurality of heterogeneous data sources comprise medical business system data, medical insurance settlement data and consumable supply chain data;

[0055] a data processing module configured to process the plurality of heterogeneous data to obtain a target data set;

[0056] a data partitioning module configured to store the target data set to a plurality of storage partitions based on a preset data model and partitioning rules to obtain a plurality of data subsets, wherein the partitioning rules support partitioning data according to at least one of a time range, a business department and a data type, and a partitioning granularity is adjusted according to a historical query frequency;

[0057] a data matching module configured to analyze filter conditions in a user query request and match a target data subset from the plurality of data subsets;

[0058] a model analysis module configured to select an analysis model from a preset model library according to an analysis target parameter in the query request and a partitioning attribute of the target data subset, analyze the target data subset and output a result;

[0059] a data interaction module configured to generate a corresponding report based on the analysis result and interact with a user through a graphical user interface.

[0060] In a third aspect, the embodiment of the present application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the data analysis method based on machine learning.

[0061] In a fourth aspect, the embodiment of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the data analysis system based on machine learning.

[0062] The data analysis method and system based on machine learning provided by the embodiments of the present application have the following beneficial effects: the present application breaks the "data island" by acquiring and fusing multi-source heterogeneous data, and processes the data to improve the data quality; secondly, the multi-source heterogeneous data is stored according to a preset rule, and the partition granularity is adjusted according to the historical query frequency, which can effectively improve the data query efficiency and reduce the difficulty of database maintenance; different data loading strategies are formulated according to the overlap degree of the filtering partition and the query time interval, which can load data according to the actual situation, avoid loading unnecessary data, and improve the data processing efficiency and resource utilization. Secondly, the data subset is intelligently matched and the analysis model is selected according to the user query request, which greatly improves the analysis efficiency and reduces human errors compared with the traditional manual code generation report; wherein, the present application filters the candidate model by analyzing the user analysis task type and data characteristics, and then optimizes the candidate model through a hyperparameter optimization algorithm, and uses the optimized model as the final analysis model, wherein the hyperparameter optimization algorithm uses a particle swarm optimization algorithm to search for the optimal hyperparameter, which can converge to the global optimal solution faster and reduce the consumption of computing resources. The weights of individual experience and social cooperation are adjusted according to the task type to adapt to the optimization needs of different data characteristics. A special fitness function is designed for different analysis tasks to ensure that the optimization target is consistent with the business needs. Through the double control of iteration number and fitness change threshold, over-optimization is avoided, and the model performance and computing cost are balanced. The number of particles is dynamically adjusted according to the data volume and computing resources to avoid resource waste while ensuring optimization effect. Finally, the report is generated based on the analysis result and interacted through the graphical user interface, which not only realizes the visualization of data presentation, but also provides deep insights, helps hospital management to quickly obtain key information, provides strong support for hospital decision-making, and improves the hospital management level. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 A flowchart of the data analysis method based on machine learning provided by an embodiment of the present application is shown in the figure;

[0064] Figure 2 A flowchart of the data analysis method based on machine learning provided by an embodiment of the present application is shown in the figure;

[0065] Figure 3 A structure block diagram of the data analysis system based on machine learning provided by an embodiment of the present application is shown in the figure;

[0066] Figure 4 A schematic block diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0067] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, technologies, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0068] For the purpose of making the objectives, technical solutions and advantages of the present application clearer, the following will combine the accompanying drawings to make a detailed description. Figures 1-3 The present application is described by specific embodiments.

[0069] Please refer to Figure 1 , Figure 1 The flowchart of the machine learning-based data analysis method provided by an embodiment of the present application is shown in the figure. The method comprises the following steps.

[0070] S101: Obtain and fuse the multi-source heterogeneous data of the target hospital, which comprises medical business system data, medical insurance settlement data and consumable supply chain data.

[0071] In the embodiment, the ETL tool is used to obtain the data from the medical business system, the medical insurance settlement system and the consumable supply chain system of the target hospital through API interface, database direct connection and file transmission. For the unstructured medical record cost description in the medical business system data, the natural language processing technology is used to extract the key information, the data cleaning is used to remove the repeated and erroneous data, the data mapping and conversion technology is used to unify the medical insurance settlement data and the consumable supply chain data in different formats into a standard format, and the data fusion is performed based on the data blood relationship and the metadata information to obtain the fused data, i.e. the multi-source heterogeneous data.

[0072] S102: Process the multi-source heterogeneous data to obtain a target data set.

[0073] In the embodiment, the multi-source heterogeneous data after fusion is reduced in data quantity through data sampling, and the feature engineering technology, including but not limited to feature selection, feature extraction and feature construction, is used to screen out the features strongly related to data analysis. The isolated forest algorithm is used to identify and process the abnormal values in the data to finally obtain the target data set.

[0074] S103: Store the target data set to a plurality of storage partitions based on a preset data model and partition rule to obtain a plurality of data subsets. The partition rule supports at least one dimension of time range, business department and data type to divide the data, and the partition granularity is adjusted according to the historical query frequency.

[0075] In the embodiment, based on a preset data model, the target data set is stored in multiple storage partitions according to at least one dimension of a time range (for example, divided by day, week, month, quarter, and year), a business department (different clinical departments, administrative departments, etc.), and a data type (income data, expenditure data, cost data, etc.) by using partitioning technologies such as hash partitioning, range partitioning, and list partitioning, to obtain multiple data subsets; at the same time, a historical query frequency analysis module is established, the query frequency and response time under different dimensions and different granularities are counted by analyzing the query operations of the user in the past period of time, the partition granularity is adjusted, and if the frequency of querying data by week in a certain period of time is high, the granularity of the week partition is correspondingly increased.

[0076] In the embodiment, a distributed storage network is constructed in combination with the blockchain technology, a unique hash value is generated for each data subset, the encrypted storage and version tracing of data are implemented, and a smart contract is introduced to guarantee the dynamic management of data access permissions; a space-time index structure is established to support the fast positioning and query of data.

[0077] S104: The filtering conditions in the user query request are analyzed, and a target data subset is matched from the multiple data subsets;

[0078] In the embodiment, regular expressions, semantic parsing, and other technologies can be used to analyze the filtering conditions in the user query request and construct a query condition tree; the partition attribute information of the multiple data subsets is traversed, and a target data subset is quickly located and matched from the multiple data subsets by using a condition matching algorithm.

[0079] S105: According to the analysis target parameter in the query request and the partition attribute of the target data subset, an analysis model is selected from a preset model library, the target data subset is analyzed, and an analysis result is output;

[0080] In the embodiment, the preset model library contains multiple machine learning basic models, for example, a long short-term memory network model for trend prediction, a decision tree model and a random forest model for classification, a clustering algorithm model for clustering analysis, a difference detection model for data comparison, and the like.

[0081] In this embodiment, according to the analysis target parameter in the query request, the corresponding analysis model is selected, the target data subset is comprehensively analyzed by using the ensemble learning algorithm, and various tasks such as prediction, classification, and correlation analysis are realized through model fusion. For the prediction task, the ensemble learning method is used, the long short-term memory network model is used as the first layer model for training, and the linear regression model is used as the second layer model to fuse the output results of the first layer model, so as to realize more accurate data trend prediction. For the classification task, the random forest model is used to determine the classification result through the voting mechanism; for the correlation analysis, the mining algorithm is used to mine the association rules between different data items.

[0082] Secondly, the multi-objective optimization framework can also be used to establish the mapping relationship between the user demand and the target function parameter, and the target function of the analysis model is dynamically adjusted according to the user demand. In addition, when the user pursues the calculation efficiency, the model with low calculation complexity is preferentially selected, and the target function is adjusted to balance the accuracy and the calculation efficiency. Through the optimization algorithm such as the particle swarm optimization algorithm and the genetic algorithm, the parameters of the analysis model are adjusted, and the accuracy, the interpretability, and the calculation efficiency are optimized. In addition, the causal inference technology can also be used to construct the data causal relationship network based on the target data subset, and the causal inference method such as the propensity score matching and the instrumental variable method is used to mine the causal relationship between the data, so as to provide the analysis result with more decision value for the user.

[0083] S106: Based on the analysis result, the corresponding report is generated, and the graphical user interface is used to interact with the user.

[0084] In this embodiment, based on the analysis result, the template engine technology is used to generate the corresponding report in combination with the user preference settings, wherein the user preference settings include the report format and the display focus, etc. The report content includes but is not limited to the text description, the statistical chart, and the data table, etc. The statistical chart includes the column chart, the line chart, and the pie chart, etc. The graphical user interface is used to provide the data visualization interaction function for the user, wherein the visualization function includes the data filtering, the chart zooming, the drilling to view the detailed data, etc. The user can also set the parameters through the interface to reinitiate the query request or adjust the analysis model, so as to realize the deep interaction with the user.

[0085] From the above, it can be concluded that the application breaks the "data island" by acquiring and fusing multi-source heterogeneous data, and processes the data to improve data quality. According to the preset rule partition storage, the partition granularity is adjusted according to the historical query frequency, which can effectively improve the data query efficiency and reduce the difficulty of database maintenance. Secondly, according to the user query request, the data subset is intelligently matched and the analysis model is selected, which greatly improves the analysis efficiency and reduces human errors compared with the traditional manual code generation report. Based on the analysis result, the report is generated and interacted through the graphical user interface, which not only realizes the visualization of the data, but also provides deep insight, helps the hospital management to quickly obtain key information, provides strong support for hospital decision-making, and improves the hospital management level.

[0086] In an embodiment of the present application, the machine learning-based data analysis method further comprises:

[0087] A real-time anomaly detection module based on autoencoder and Transformer is constructed to continuously monitor the trend of data changes. When abnormal fluctuations are detected, a warning mechanism is automatically triggered, and the attention mechanism is used to locate the root cause of the anomaly.

[0088] A financial digital twin model of the target hospital is constructed to map the financial situation of the target hospital. By simulating different business scenarios, the user can perform risk assessment and strategic planning.

[0089] In an embodiment of the present application, the multi-source heterogeneous data of the target hospital is obtained, comprising:

[0090] The structured data and unstructured data of the target hospital internal system and external data platform are obtained through the API interface;

[0091] The natural language processing model is used for entity recognition and relationship extraction of unstructured data to generate structured processing data;

[0092] The structured data and structured processing data are aligned in data mode and processed in format uniformity;

[0093] The aligned data is weighted and fused based on the weight coefficients of each data source of the target hospital to generate multi-source heterogeneous data.

[0094] In this embodiment, the data of the target hospital internal system and external data platform is obtained through various types of API interfaces. For internal systems such as medical business systems and medical insurance settlement systems, RESTful API or SOAP API is used to obtain structured data such as patient treatment cost details and medical insurance reimbursement records in a timely or real-time manner according to the agreed data transmission protocol of the system. For external data platforms such as government fiscal policy release platforms and industry data statistics platforms, OAuth authenticated API interfaces are used to pull structured and unstructured data such as macro fiscal policy files and industry financial indicators after obtaining permission. At the same time, an API interface monitoring module is set up to monitor the interface call status, response time and data return integrity in real time. When the interface is abnormal, the standby interface is automatically switched or the retry operation is performed, and the abnormal log is recorded.

[0095] In this embodiment, a natural language processing model based on deep learning is used to process unstructured data. For medical record cost description documents in the medical business system, the text is first segmented into word sequences using word segmentation technology, then input into the BERT pre-trained language model for feature extraction to capture word context semantic information, then the long short-term memory network is used for bidirectional learning of the features, and finally the CRF layer is used for entity recognition and relationship extraction to extract entities such as drug names, examination items and cost amounts and their cost association relationships, and generate structured processing data. For unstructured text such as government fiscal policy documents, named entity recognition technology is used to extract key entities such as subsidy objects and subsidy standards involved in the policy, and relationship extraction technology is used to determine the logical relationship between policy clauses to form structured data.

[0096] In this embodiment, a data mode mapping table is established to analyze the field meaning, data type and semantic relationship of structured data and structured processing data. For data with the same meaning but different field names, such as "patient ID" in the internal system and "medical record number" in the external platform, unified naming is performed through the mapping table. For inconsistent data types, such as date fields stored in "YYYY-MM-DD" and "DD / MM / YYYY" formats in different systems, data conversion functions are used to unify them into a standard date format. This embodiment uses data standardization technology to normalize or standardize numerical value data so that the data is in the same numerical value range. For classification data, one-hot encoding or label encoding is used for conversion.

[0097] In the embodiment, the weight coefficients of each data source of the target hospital are determined by combining subjective and objective methods. In the subjective aspect, experts and business personnel score according to the importance of the data source in data analysis. In the objective aspect, the information entropy of each data source is calculated by using the entropy weight method. The smaller the information entropy, the greater the amount of information provided by the data source, and the higher the weight. By combining the subjective and objective weights, the analytic hierarchy process is used to determine the final weight coefficients of each data source. Based on the determined weight coefficients, the aligned data is weighted and fused. For data with the same attribute, such as medical income data in different data sources, the weighted average value is calculated according to the weight proportion. For data with complementary attributes, such as the cost detail data of the internal system and the industry cost reference data of the external platform, the data is directly integrated to generate multi-source heterogeneous data. In the data fusion process, a data conflict monitoring mechanism is established. When the data of different data sources conflicts, the real-time data of the internal system is processed according to the preset priority rules, and the conflict processing process is recorded.

[0098] In one possible embodiment, the formula for weighted fusion of the aligned data based on the weight coefficients of each data source of the target hospital is:

[0099]

[0100] wherein, is the multi-source heterogeneous data, is the weight coefficient of the i th data source, is the aligned data of the i th data source, is the element-wise multiplication, is the feature confidence mask matrix, used to adjust the fusion weight of each feature value; when the confidence is ≥ θ (confidence threshold), ; when the confidence is < θ (confidence threshold), . In one embodiment of the present application, the method for determining the weight coefficients of each data source comprises:

[0101] In one embodiment of the present application, the method for determining the weight coefficients of each data source comprises:

[0102] Obtaining evaluation index data of each data source of the target hospital, the evaluation index data including: data update frequency index, historical accuracy rate index, and data provider reputation level index;

[0103] Calculating the weight coefficients of each data source according to the evaluation index data and a preset first weight calculation formula.

[0104] ​In this embodiment, data update logs are obtained from each data source system through an API interface, the timestamp of each data update is recorded, the time interval between adjacent two updates is calculated, and the average update frequency within a certain time period is counted. For real-time updated data sources, the highest update frequency score is given; for daily, weekly, and monthly updated data sources, the corresponding scores are calculated according to their update periods respectively.

[0105] In this embodiment, a certain proportion of data samples are periodically extracted from each data source and compared with the standard data set verified by artificial verification. For numerical data, absolute error and relative error are calculated; for classification data, accuracy, recall rate, F1 value and other indicators are calculated. According to the comparison result, the average accuracy of each data source in the historical period is counted as the historical accuracy indicator.

[0106] The relevant qualification information of the data provider is collected, including the industry certification of the data provider, the operation time, the historical data quality feedback record, etc.; the credit evaluation model is used to quantitatively score the credit of the data provider. For data provided by government departments, authoritative industry organizations, etc., a higher credit level score is given; for commercial data suppliers, the credit level is comprehensively evaluated according to the service quality, data accuracy historical record and other factors.

[0107] For the data update frequency indicator, it is converted to a value between 0 and 1, the higher the update frequency, the closer the value to 1. For example, the score of real-time updated data source is 1, the score of daily updated data source is 0.8, the score of weekly updated data source is 0.5, and the score of monthly updated data source is 0.2.

[0108] For the historical accuracy indicator, the accuracy value calculated is directly used as the score, for example, if the historical accuracy of a data source is 95%, the score is 0.95.

[0109] For the data provider credit level indicator, according to the score result of the credit evaluation model, it is mapped to a value between 0 and 1. For example, the score of data source with “excellent” credit level is 0.9, the score of “good” is 0.7, the score of “general” is 0.5, and the score of “poor” is 0.3.

[0110] This embodiment can also monitor the changes of the evaluation indicator data of each data source in real time; when the data update frequency of a certain data source decreases significantly, the historical accuracy fluctuates or the credit level of the data provider changes, the recalculation of the weight coefficient is triggered. According to the data usage feedback and the change of business demand, the weight parameters are adjusted periodically (such as every quarter). For example, with the advancement of hospital informationization construction, the accuracy and real-time performance of internal data sources are continuously improved, and the weight coefficient of internal data sources can be increased.

[0111] In one possible embodiment, the first weight calculation formula is:

[0112]

[0113] wherein, is a first adjustment coefficient, the initial value of which is set by an expert and is subsequently automatically optimized according to model feedback; is an accuracy amplification factor to ensure that high-accuracy data obtains a higher weight; is a second adjustment coefficient, the initial value of which is set by an expert and is subsequently automatically optimized according to model feedback; is an update frequency index of the i th data source; is a historical accuracy index of the i th data source; is a reputation level index of the i th data source; is a minimum reputation level benchmark value; is a third adjustment coefficient, the initial value of which is set by an expert and is subsequently automatically optimized according to model feedback; prevents high-frequency data from monopolizing the weight. In one embodiment of the present application, after calculating the weight coefficients of each data source according to the evaluation index data and the preset first weight calculation formula, the method further comprises: in response to the data update frequency index being less than a first update frequency threshold, reducing the weight coefficient of the data source by a first step size;

[0114] in response to the data update frequency index being less than a second update frequency threshold and greater than or equal to the first update frequency threshold, reducing the weight coefficient of the data source by a second step size; wherein the first update frequency threshold is less than the second update frequency threshold, and the first step size is greater than the second step size.

[0115] In the present embodiment, based on historical data update frequency statistical analysis, a time series prediction model is established to predict the update frequency trend of each data source in the future period of time; the first update frequency threshold and the second update frequency threshold are determined in combination with the timeliness requirement of the business demand and the data analysis task. For example, for medical insurance settlement data with high real-time requirement, the first update frequency threshold is set to be updated once per hour, and the second update frequency threshold is set to be updated once per day. The present embodiment can also periodically collect user feedback and business department requirements to evaluate the rationality of the current threshold setting, and adjust the threshold size according to the feedback result. When the data analysis task has a higher requirement for data real-time, the update frequency threshold is correspondingly reduced.

[0116]

[0117]

[0118] ​​​In the embodiment, when the data update frequency index of a data source is less than the first update frequency threshold, a fast weight adjustment mechanism is started to reduce the weight coefficient of the data source by a first step length. For example, if the first step length is 0.1 and the original weight coefficient of the data source is 0.6, the adjusted weight coefficient becomes 0.5. At the same time, a weight adjustment log is recorded, including the adjustment time, the weight values before and after the adjustment, the update frequency value triggering the adjustment, and the like.

[0119] When the data update frequency index is less than the second update frequency threshold and greater than or equal to the first update frequency threshold, a slow weight adjustment mechanism is started to reduce the weight coefficient of the data source by a second step length. For example, if the second step length is 0.05 and the original weight coefficient of the data source is 0.6, the adjusted weight coefficient becomes 0.55.

[0120] In the embodiment, a lower limit value of the weight coefficient is set to prevent the weight coefficient from being excessively reduced. When the adjusted weight coefficient is lower than the lower limit, the weight coefficient is fixed to the lower limit value, ensuring that the data source still has a certain reference value even if the update frequency is very low.

[0121] The embodiment also constructs a weight adjustment buffer period. During the buffer period, the update frequency change of the data source is continuously monitored. If the data source restores the normal update frequency during the buffer period, the weight coefficient of the data source is gradually restored; if the update frequency still does not meet the standard, the adjusted weight coefficient is maintained.

[0122] In the embodiment, when the weight coefficient of a data source is adjusted due to the update frequency, the weight coefficients of other data sources are recalculated to keep the sum of the weight coefficients of all data sources to be 1. The weight coefficients of other data sources are adjusted by using the proportional scaling method to ensure that the adjusted weight distribution is reasonable.

[0123] In the embodiment, the data fusion quality before and after the weight adjustment and the accuracy of the analysis result are compared. If it is found that the quality of the analysis result decreases after the adjustment, the update frequency threshold and the step length parameters are adjusted in time to ensure the effectiveness of the weight adjustment strategy.

[0124] In an embodiment of the present application, based on a preset data model and partition rules, a target data set is stored in a plurality of storage partitions, including:

[0125] The target data set is converted into a structured data set according to the preset data model;

[0126] The time partition granularity is determined based on the historical query frequency, and the structured data set is divided into a primary storage partition according to the time range dimension;

[0127] The data records in the primary storage partition are divided into a secondary sub-partition according to the business department dimension;

[0128] According to the data type dimension, the data records in the secondary sub-partition are divided into three-level sub-partitioning;

[0129] A partition metadata index table is established to record the time range, business department code and data type code of each partition;

[0130] The data subsets after three-level division are distributed and stored in corresponding physical storage nodes to generate multiple data subsets.

[0131] In this embodiment, the target data set is structured and converted according to a preset data model. For the nested expense detail data in the medical business system, the multi-layer nested JSON or XML format data is expanded into a two-dimensional table structure through data flattening processing; for the redundant fields existing in the medical insurance settlement data, the data standardization technology is used to remove duplicate data storage, and the data structure is ensured to meet the standard of the preset model. In the conversion process, the data quality checking tool is used to check the field type consistency, primary key uniqueness, and foreign key association accuracy, etc. If the data format is found to be incorrect or incomplete, the data cleaning process is automatically triggered to modify or mark the abnormal data.

[0132] In this embodiment, the time range involved in the user query operation in the past 12 months is counted, and the query frequency proportion of different time granularity (day, week, month, quarter, year) is calculated. For example, if it is found that the frequency of quarterly query reaches 40%, which is significantly higher than other granularities, then the quarter is taken as the initial time partition granularity. Combined with the data growth trend prediction, the exponential smoothing method or linear regression model is used to estimate the future data volume. If it is predicted that the data volume will increase significantly in a certain period of time, the time partition granularity is refined in advance, such as dividing the quarterly partition into monthly partition, to avoid the influence of the large data volume in a single partition on the query performance.

[0133] According to the determined time partition granularity, the structured data set is divided into primary storage partitions according to the time range dimension. A unique time identifier is assigned to each primary partition, such as "2025Q1" and "2025Q2", and a time index is established in the partition to speed up the time range query operation.

[0134] In this embodiment, a business department coding system is established to uniformly code clinical departments (such as cardiology department, respiratory department), medical technology departments (such as radiology department, laboratory department), administrative departments (such as finance department, medical department) and the like of the target hospital. The coding rule adopts a hierarchical structure, for example, “01-cardiology department-0101-outpatient” and “01-cardiology department-0102-inpatient”, to ensure clear classification of department data. In specific implementation, each data record in the primary storage partition is traversed, the business department information field contained in the data is extracted, and the data is divided into the corresponding secondary sub-partition according to the coding system. For data across departments, such as joint diagnosis and treatment project cost data, the data is divided according to the main responsible department, and an auxiliary identification field is added in the data record to indicate the other departments involved. If it is found in this embodiment that the proportion of the data volume of some departments is too high, resulting in a decrease in query performance, the secondary sub-partition of these departments is further subdivided, such as a tertiary sub-partition according to different wards or project groups under the department.

[0135] In this embodiment, data type classification standards are defined to divide data into income type (medical income, medical insurance income, financial subsidy income), expenditure type (personnel expenditure, consumable expenditure, equipment procurement expenditure), cost type (direct cost, indirect cost) and the like. A unique code is assigned to each data type, such as “01-medical income” and “02-personnel expenditure”.

[0136] The data records in the secondary sub-partition are divided into a tertiary sub-partition according to the data type field. A data type mapping table is established, and when the data type field value is ambiguous or not standardized, it is mapped to the correct data type partition through semantic analysis and rule matching. Differentiated storage strategies are adopted for different types of data. For high-frequency query income data, a columnar storage format is adopted to improve the aggregation query efficiency; for relatively low-frequency query historical cost data, a row storage format is adopted to save storage space.

[0137] In this embodiment, the established partition metadata index table includes time range field (such as start time, end time), business department coding field, data type coding field, partition physical storage path field, data record quantity field and the like. In this embodiment, the statistical information such as the data record quantity in the index table can be automatically updated by scanning the data in each partition.

[0138] In this embodiment, a composite index can also be established for the index table, with time range, business department code and data type code as index keys, to speed up the speed of positioning the target partition according to the user query condition. At the same time, the distributed hash table (DHT) technology is used to store the index table data on multiple nodes to improve the query and maintenance performance of the index table.

[0139] The application can provide an index table visual management interface, allowing an administrator to manually modify or add partition metadata information, such as adjusting the partition time range, modifying the department code correspondence, and the like, and automatically synchronizing to each storage node after the operation is completed.

[0140] In the embodiment, a distributed file system or a distributed database is used as a storage carrier, and according to the metadata information of each partition, the data subsets completed by three-level division are stored to the corresponding physical storage nodes. In the storage process, data redundancy technologies such as copy replication or erasure coding are used to ensure the reliability of the data, and when a certain storage node fails, the data can be obtained from other copy nodes. The load condition and storage space utilization of each storage node are dynamically monitored, and when a certain node has too high load or insufficient storage space, a data migration mechanism is automatically triggered to migrate part of the data subsets to a node with lower load, so as to realize balanced allocation of storage resources.

[0141] In an embodiment of the application, the time partition granularity is determined based on the historical query frequency, including:

[0142] The query frequency of different time intervals in the historical query request is counted.

[0143] According to the mapping relationship between the query frequency and the time interval, the day-level, month-level or year-level partition granularity is selected.

[0144] In the embodiment, a query log collection system is established to collect the query requests initiated by users in real time, record the query time, the time interval in the query condition (such as “2025-01-01 to 2025-01-31”), the data volume involved in the query, the query response time and the like.

[0145] The historical query log is cleaned and standardized to remove invalid query records (such as requests with incorrect query parameters), unify the representation format of the time interval, and convert different formats of the time range (such as “the first quarter of 2025” into “2025-01-01 to 2025-03-31”).

[0146] In the embodiment, key features are extracted from the cleaned query log, including the time interval length, the query frequency, the query heat (calculated based on the query response time and resource consumption), and the like, to form a structured historical query data set.

[0147] In the embodiment, a time interval bucket division rule is established to divide the time axis into buckets of different lengths, such as day buckets (each bucket represents a day), week buckets (each bucket represents a week), month buckets (each bucket represents a month), quarter buckets (each bucket represents a quarter), and year buckets (each bucket represents a year).

[0148] Count the query frequency in each time bucket, and calculate the query frequency distribution under different time granularities (day, week, month, quarter, and year). For example, count how many times the query appears in the past year with a time interval of "day" and how many times the query appears with a time interval of "month".

[0149] Construct a query frequency-time interval matrix, with the horizontal axis representing the time granularity (day, week, month, quarter, and year) and the vertical axis representing the query frequency. The mapping relationship between the two can be visualized by a heat map or a scatter plot. Use association rule mining algorithms to find strong association rules between different time granularities and query frequencies.

[0150] In this embodiment, the query frequency weight of each time granularity is calculated, and a partition granularity selection decision tree model is established. The input factors include the query frequency weight of each time granularity, data volume prediction value, and storage cost, and the output is the optimal partition granularity. The division rule of the decision tree is based on cost-benefit analysis, and the balance between query performance improvement and storage overhead increase is considered comprehensively.

[0151] Set the partition granularity selection threshold. When the query frequency weight of a certain time granularity exceeds the threshold, and the average query response time under that time granularity meets the business requirements, the time granularity is selected as the partition granularity. If the weights of multiple time granularities are close, the middle granularity is selected as a compromise. This embodiment tracks the data volume growth and query frequency trend of each time partition in real time. When it is found that the data volume of a certain time partition exceeds the preset threshold or the query frequency proportion of a certain time granularity changes significantly, the partition granularity adjustment process is triggered. This embodiment can also predict the query frequency trend of each time granularity in the future based on a time series prediction model, and adjust the partition granularity in advance to adapt to the changes in query mode. For example, if it is predicted that the monthly query frequency will increase significantly in the future, some quarterly partitions will be split into monthly partitions.

[0152] In an embodiment of the present application, according to the mapping relationship between query frequency and time interval, day-level, month-level, or year-level partition granularity is selected, including:

[0153] When the time interval meets the first time interval condition and the corresponding query frequency is greater than the first query frequency threshold, day-level partition granularity is used;

[0154] When the time interval meets the second time interval condition and the corresponding query frequency is greater than the second query frequency threshold, month-level partition granularity is used;

[0155] When the time interval meets the third time interval condition and the corresponding query frequency is greater than the third query frequency threshold, year-level partition granularity is used.

[0156] In this embodiment, the first time interval condition is that when the length of the time interval is less than or equal to a preset first length threshold, the first time interval condition is determined to be met. For example, the first length threshold is 5 days, the query condition is "2025-01-01 to 2025-01-05", the length of the time interval is 5 days, and the first time interval condition is met.

[0157] The second time interval condition is that when the length of the time interval is greater than the first length threshold and less than or equal to a preset second length threshold, the second time interval condition is determined to be met. For example, the second length threshold is 90 days, the query condition is "2025-01-01 to 2025-03-31", the length of the time interval is 90 days, and the second time interval condition is met.

[0158] The third time interval condition is that when the length of the time interval is greater than the second length threshold, the third time interval condition is determined to be met. For example, the query condition is "2025-01-01 to 2025-12-31", the length of the time interval is 365 days, and the third time interval condition is met.

[0159] In this embodiment, according to the seasonal fluctuation characteristics of hospital data, the time interval threshold can be reduced during the business peak period to increase the proportion of fine-grained partitions; during the business trough period, the threshold can be relaxed to reduce the number of partitions and reduce storage costs.

[0160] In this embodiment, based on the statistical distribution characteristics of historical query data, the query frequency threshold is determined using a box plot or a 3σ principle. For example, the median and interquartile range of all time interval query frequencies are calculated, and the first query frequency threshold is set to the median plus 1.5 times the interquartile range.

[0161] In this embodiment, an association model between the query frequency threshold and the data volume is established. When the data volume at a certain time granularity exceeds a preset capacity threshold, the query frequency threshold of the time granularity is increased accordingly to avoid excessive partitions due to excessive data volume.

[0162] In this embodiment, when a time interval meets multiple time interval conditions at the same time, a priority rule is used to determine the final partition granularity. The priority order is: daily partition granularity > monthly partition granularity > annual partition granularity. For example, if the length of a time interval is 30 days and meets both the first and second time interval conditions, and the corresponding query frequencies exceed their respective thresholds, the daily partition granularity is preferred.

[0163] In this embodiment, the priority rule can also be adjusted; when storage resources are sufficient, the priority of fine-grained partitions is increased; when storage resources are tight, the priority of fine-grained partitions is reduced.

[0164] In this embodiment, when the user query involves multiple time granularities, both daily and monthly data are queried simultaneously, and a hybrid partitioning strategy is adopted, i.e., fine-grained partitioning is used for the time range of high-frequency queries, and coarse-grained partitioning is used for the time range of low-frequency queries. For example, daily partitioning is used for the data of the last month, and monthly partitioning is used for historical data. When a significant change in the mapping relationship between query frequency and time interval is detected (for example, the query frequency of a certain time granularity suddenly decreases by 80%), a partition granularity reevaluation process is triggered, and the partitioning strategy is adjusted according to the new mapping relationship.

[0165] In an embodiment of the present application, the filtering conditions in the user query request are parsed, and the target data subset is matched from multiple data subsets, including:

[0166] The filtering conditions in the user query request are parsed to obtain the query time interval, the query department set, and the query data type code;

[0167] The partition metadata index table is queried to filter the partitions that meet the filtering conditions, and the filtered partitions are obtained;

[0168] The filtering conditions include: the partition time range overlaps with the query time interval, the partition business department code belongs to the query department set, and the partition data type code matches the query data type code;

[0169] According to the overlap degree of the time range of the filtered partition and the query time interval, the loaded data is determined, and the loaded data is merged into the target data subset.

[0170] In this embodiment, the natural language processing technology is combined with regular expressions to parse the user query request. For structured query statements, the time interval, department name, and data type keywords are directly extracted; for natural language description query requests, such as "view the medical income data of the cardiology department in the first half of 2025", the named entity recognition model is used to extract the key entities "the first half of 2025", "cardiology department", and "medical income", and then the natural language is converted into structured query conditions through semantic analysis to obtain the accurate query time interval, query department set (the department code set corresponding to "cardiology department"), and query data type code (the code corresponding to "medical income").

[0171] In this embodiment, when the parsed keywords have semantic ambiguity or spelling errors, synonym libraries and similar word matching algorithms are used for correction and supplementation. For example, if the user inputs "cardiology lesson", it is automatically matched to "cardiology department"; if the query data type keyword is incomplete, such as only "income" is input, it is inferred and supplemented to "medical income" according to the context and historical query mode, ensuring the accuracy of the parsed results.

[0172] In the embodiment, a multi-level index structure is established for the partition metadata index table. In addition to the existing composite index of time range, business department code and data type code, a prefix index of time range and an inverted index of business department code are additionally established for high-frequency query scenarios. In query, the prefix index is used to quickly locate the approximate time range partition, the inverted index is used to filter the partitions corresponding to the department, and finally the composite index is used for accurate matching, thereby improving the query efficiency of the index table.

[0173] In the embodiment, for the time range filtering, an interval tree data structure is used to organize and manage the partition time range. The time range of each partition is taken as a node of the interval tree. In query, the interval tree is used to quickly find the partitions that overlap with the query time interval. Compared with the traditional linear scanning method, the time complexity can be significantly reduced.

[0174] For the filtering of the business department code, when the query department set contains multiple departments, bit operation is used to speed up the matching process. A unique binary bit is assigned to each business department code, the query department set is converted into a binary mask, and the bit AND operation is used to quickly determine whether the partition business department code belongs to the query department set, thereby improving the filtering efficiency.

[0175] In the data type code matching, a hierarchical mapping relationship of the data type code is established. When the query data type code is a parent node code, all the partitions corresponding to the child node codes of the parent node code are automatically matched. For example, if the query data type code is "income type", all the partitions containing "medical income", "medical insurance income" and other income type subtypes are automatically filtered, thereby enhancing the flexibility of the filtering.

[0176] In the embodiment, based on the overlap degree of the time range of the filtered partition and the query time interval, a dynamic data loading strategy is used. When the overlap degree is greater than a second overlap threshold, the entire partition data is directly loaded; when the overlap degree is low, the data is loaded in a paging manner or on demand, and only the data overlapping with the query time interval is loaded, thereby reducing unnecessary data transmission and memory occupation.

[0177] In an embodiment of the present application, the data to be loaded is determined according to the overlap degree of the time range of the filtered partition and the query time interval, including:

[0178] When the overlap degree is less than a first overlap threshold, the partition is skipped;

[0179] When the overlap degree is greater than or equal to the first overlap threshold and less than a second overlap threshold, the data overlapping with the query time range and the partition time range is loaded;

[0180] When the overlap degree is greater than or equal to the second overlap threshold, the data of the partition is loaded.

[0181] In this embodiment, based on historical query data, the relationship between data loading efficiency and query response time under different overlap degrees is analyzed, and an overlap degree-performance index prediction model is established. With the shortest query response time and the smallest data loading resource consumption as the goal, the optimal first overlap threshold and second overlap threshold are calculated. For example, through the analysis of the past 1000 queries, when the overlap degree reaches 30, the data loading efficiency begins to improve significantly, and when it reaches 70%, it approaches saturation, so the first overlap threshold can be initially set to 30% and the second overlap threshold to 70%. The overlap threshold can also be manually adjusted in this embodiment.

[0182] When the partition data volume is large, the first overlap threshold can be increased to reduce unnecessary data loading; when the data volume is small, the first overlap threshold can be reduced to avoid incomplete data due to excessive skipping of partitions. At the same time, the threshold is dynamically adjusted according to the data update frequency. For real-time updated data, the threshold is reduced to ensure that the latest data can be loaded in time.

[0183] In this embodiment, when the overlap degree is less than the first overlap threshold, the partition information is recorded in the "low correlation partition list" in addition to skipping the partition. If the partition is skipped multiple times due to low overlap in subsequent queries, the system automatically archives or compresses the data of the partition to reduce storage resource occupation. At the same time, the "low correlation partition list" is checked at a certain period (such as every month), and if the partition data changes (such as data update), the overlap degree of the partition with the query time interval is re-evaluated to avoid misjudgment.

[0184] When the overlap degree is greater than or equal to the first overlap threshold and less than the second overlap threshold, an incremental data loading method is used. Using the time index of the partition, the data segment overlapping the query time range and the partition time range is quickly located, and only this part of data is loaded. For the loaded data, data format conversion and preprocessing are performed to make it meet the input requirements of the subsequent analysis model. In order to improve the loading speed, this embodiment uses multi-thread or distributed loading technology to load multiple overlapping data of partitions in parallel.

[0185] When the overlap degree is greater than or equal to the second overlap threshold, all data of the partition is loaded,

[0186] In the embodiment, a hash check code is generated for the loaded data, and after the data loading is completed, whether the data is complete in the transmission and loading process is verified by comparing the hash values. If the hash values are inconsistent, the data is automatically reloaded, and error logs are recorded, including error partition information, error time, error cause, etc., to facilitate subsequent troubleshooting. For overlapping data loaded, the integrity of the boundary data is checked. For example, when loading data of two partitions overlapping, it is ensured that the data of the overlapping boundary is neither repeatedly loaded nor omitted. The embodiment can also select appropriate analysis models and parameters according to the amount and characteristics of the loaded data. For example, when the amount of loaded data is small, a model with lower computational complexity is preferred; when the amount of data is large, a distributed computing framework is used to run the analysis model to ensure efficient completion of the analysis task.

[0187] In an embodiment of the present application, multi-source heterogeneous data is processed to obtain a target data set, comprising:

[0188] An outlier is detected and corrected by an isolated forest algorithm to generate first data;

[0189] Noise data in the first data is identified and removed by a clustering algorithm;

[0190] The noise-removed data is processed for missing values by a statistical method to generate cleaned data;

[0191] Based on the hospital operation analysis target, a feature data set is constructed from the cleaned data;

[0192] According to the feature importance and correlation analysis, the features are screened to generate a target data set.

[0193] In the embodiment, the numerical data in the multi-source heterogeneous data is input into the isolated forest algorithm model. In the model training stage, multiple isolated trees are constructed by randomly selecting sample data, and the number of trees and the size of the sub-sample are set. Each isolated tree starts from the root node, randomly selects a feature and a split point of the feature, divides the sample data into left and right child nodes, and continues until each leaf node contains only one sample or reaches the preset tree depth. The abnormal score is calculated according to the path length of the sample in the tree, and if the abnormal score of a sample is higher than the preset abnormal threshold, it is determined as an outlier. For outliers, a correction method based on statistical distribution is used to calculate the mean and standard deviation of the feature data, and the outliers are replaced by the nearest normal value within the range of mean ± 3 times standard deviation to generate first data.

[0194] For the first data, a clustering algorithm is used to identify noise data. The neighborhood radius is set to the minimum number of samples, and each sample point in the data set is traversed to calculate the number of samples within the neighborhood radius. If the number of neighborhood samples of the sample point is less than the minimum number of samples and the sample point does not belong to any formed cluster, it is determined to be noise data. The identified noise data is removed from the first data to reduce the interference of noise on subsequent analysis.

[0195] In this embodiment, for the data after removing noise, a statistical method is used for missing value processing. For missing values of numerical data, if the missing proportion is below the first missing range, the mean or median is used for filling; if the missing proportion is within the second missing range, a multiple imputation method is used to construct a regression model based on other related features to predict the missing values; if the missing proportion is within the third missing range, the feature column is directly deleted. For missing values of classification data, the mode is used for filling. Among them, the third missing range is greater than the second missing range, and the second missing range is greater than the first missing range.

[0196] In this embodiment, based on the hospital operation analysis target, such as medical income prediction, cost control analysis, etc., relevant fields are extracted from the cleaned data to construct a feature data set. For example, in order to analyze the influencing factors of medical income, the fields of patient age, department, treatment project, medical insurance type, hospitalization days, etc. are extracted as features; in order to perform cost control analysis, the fields of consumable name, purchase quantity, purchase unit price, use department, use time, etc. are extracted. At the same time, some features are derived, such as calculating the per capita medical expense, consumable usage rate, etc. new features, to enrich the feature dimension.

[0197] The random forest algorithm is used to calculate the feature importance, and by randomly dividing the training set and test set multiple times, the contribution of each feature to the model accuracy is calculated. The higher the contribution, the more important the feature. The Pearson correlation coefficient is used to analyze the correlation between features, and a correlation threshold is set to remove redundant features with high correlation. According to the feature importance and correlation analysis results, the most valuable features for the analysis target are selected to generate a target data set.

[0198] In an embodiment of the present application, the feature data set includes: a feature reflecting the cost-benefit ratio of the department, a feature reflecting the difference between medical insurance payment and actual charges, and a feature reflecting the correlation between consumable use and income.

[0199] Among them,

[0200] By comparing the payment details in the medical insurance settlement data with the charge details in the medical business system, a feature reflecting the difference between medical insurance payment and actual charges is constructed.

[0201] By associating the consumable use records in the consumable supply chain data with the income data in the medical business system, a feature reflecting the correlation between consumable use and income is constructed.

[0202] Calculate the cost-benefit ratio characteristics of each business department, including the proportion of personnel costs, equipment depreciation rate, and revenue profit margin.

[0203] In this embodiment, the payment details in the medical insurance settlement data and the charging details in the medical business system are cleaned and formatted to ensure that key information such as patient identification and treatment item codes are consistent. Using a data matching algorithm, the medical insurance payment details are associated with the corresponding medical charging details, with the patient's unique identifier (e.g., ID card number or medical insurance card number) and the treatment item code as the matching key.

[0204] For each successfully matched record, the difference between the medical insurance payment and the actual charge, as well as the difference rate, are calculated. For multiple medical records of the same patient at different times, statistics are summarized by treatment cycle (e.g., monthly or quarterly) to obtain the total difference between medical insurance payment and actual charge, the average difference rate, and other characteristics for each cycle.

[0205] Time series analysis was used to analyze the changing trend of the difference between medical insurance payments and actual charges over time, extracting trend features and seasonal characteristics to enrich the dimensions of the difference features. Simultaneously, an anomaly detection mechanism was established; when the difference rate within a certain time period exceeds a certain threshold, the data is marked as abnormal and manually reviewed to ensure the accuracy of the feature data.

[0206] In this embodiment, the consumable usage records extracted from the consumable supply chain data include information such as consumable name, quantity used, department used, and usage time; the revenue data extracted from the medical business system includes information such as revenue items, revenue amount, corresponding department, and time. Using a data association algorithm, with department and time as the association dimensions, the consumable usage records are associated with the revenue data to establish a consumable usage-revenue association table.

[0207] Based on association rule mining algorithms, we can discover association rules between different combinations of consumables and specific revenue items. For example, we can find rules that significantly increase revenue for a certain type of surgery when using a combination of consumables A and B, and transform these rules into feature representations for subsequent analysis and decision support.

[0208] In this embodiment, the personnel cost ratio is calculated as follows: Personnel salaries, benefits, training, and other expense data for each department are obtained from the hospital's human resource management system and financial accounting system, and these are used as personnel costs. Simultaneously, the total revenue data for each department is also obtained. The personnel cost ratio is calculated using the formula "Personnel Cost Ratio = Personnel Cost / Total Department Revenue × 100%". To more accurately reflect the personnel cost structure, personnel types are further subdivided (e.g., doctors, nurses, administrative staff), and the cost ratio for each personnel type is calculated separately.

[0209] In this embodiment, the equipment depreciation rate is calculated: according to the purchase amount, the expected service life and the used life of the equipment in each department of the hospital fixed asset management system, the linear depreciation method is used to calculate the equipment depreciation amount. The formula is "annual depreciation amount = (purchase amount - expected net salvage value) / expected service life", wherein the expected net salvage value can be set as a fixed proportion according to the type of equipment. The equipment depreciation rate of each department is calculated, and the formula is "equipment depreciation rate = depreciation amount / equipment purchase amount x 100%".

[0210] In this embodiment, the income profit rate is calculated: the net profit and total income data of each department are obtained from the financial data report, and the income profit rate characteristics are calculated by the formula "income profit rate = net profit / department total income x 100%". According to the business characteristics and market environment of the department, the income profit rate is compared and analyzed horizontally and vertically. The horizontal comparison is made with similar departments of other hospitals in the same industry, and the vertical analysis is made on the income profit rate change trend of the department in different time periods. The key factors affecting the income profit rate are found out, and these analysis results are converted into auxiliary features for more comprehensive evaluation of the cost-benefit ratio of the department.

[0211] In an embodiment of the present application, according to the analysis target parameters in the query request and the partition attributes of the target data subset, an analysis model is selected from a preset model library, including:

[0212] The analysis task type parameters in the query request are parsed, including task type, analysis index and business constraint conditions;

[0213] Based on the analysis task type parameters and the key data characteristics of the target data subset, a candidate model set is selected from the preset model library; the key data characteristics include at least one of data volume, feature dimension, time series characteristics and label distribution;

[0214] The performance of the candidate model is evaluated using sample data of the target data subset, and the evaluation index is set according to the task type;

[0215] If any key evaluation index is lower than a preset threshold, the candidate model is optimized by a hyperparameter optimization algorithm, and the optimized model is used as the final analysis model.

[0216] In this embodiment, natural language processing technology is used to perform semantic analysis on the query request to identify the analysis target parameters therein. For structured query statements, the prediction type, analysis index and business constraint conditions are directly extracted; wherein the prediction type includes: income prediction and cost prediction; the analysis index includes: profit rate and growth rate; the business constraint conditions include: time window and department range;

[0217] For natural language description queries, key entities and intentions are extracted using a named entity recognition model, which are converted into structured analysis target parameters.

[0218] According to the combination of analysis target parameters, the query request is divided into different analysis types:

[0219] Prediction type: including time series prediction, regression prediction, etc., corresponding to prediction type parameters such as "income prediction", "cost prediction", etc.

[0220] Classification type: including risk level classification, medical insurance payment type classification, etc., corresponding to prediction type parameters such as "high risk identification", "medical insurance type classification", etc.

[0221] Clustering type: including patient group clustering, department performance clustering, etc., corresponding to prediction type parameters such as "patient grouping" and "department ranking".

[0222] Anomaly detection type: including medical insurance fraud detection, financial data anomaly identification, etc., corresponding to prediction type parameters such as "abnormal transaction detection" and "cost anomaly identification".

[0223] The embodiment filters candidate models by analyzing user analysis task types and data characteristics, avoids manual trial and error, and improves model adaptability. Sample data is used to evaluate the performance of the candidate model to ensure the effectiveness of the selected model on actual data and reduce model bias. When the model performance is not up to standard, automatically trigger hyperparameter optimization to improve model accuracy and generalization ability to adapt to different business scenarios. According to different analysis tasks, set special evaluation indicators and fitness functions to ensure the pertinence of model optimization direction.

[0224] In an embodiment of the present application, a candidate model set is filtered based on analysis task type parameters and key data characteristics, including:

[0225] If the analysis task type parameter is prediction type and the key data characteristics contain time series characteristics, then filter the time series prediction model in the model library;

[0226] If the analysis task type parameter is classification type and the feature dimension in the key data characteristics is greater than the first preset dimension (50 dimensions), then filter the classification model in the model library that supports high-dimensional features;

[0227] If the analysis task type parameter is clustering type and the data volume in the key data characteristics is greater than the first sample number threshold (1 million samples), then filter the distributed clustering model in the model library;

[0228] If the analysis task type parameter is anomaly detection type and the label distribution in the key data characteristics is imbalanced, then filter the anomaly detection model in the model library that is suitable for non-balanced data.

[0229] In an embodiment of the present application, the performance evaluation index is set according to the task type, including:

[0230] When the task type is prediction type, the evaluation index is root mean square error or mean absolute percentage error;

[0231] When the task type is classification type, the evaluation index is F1 score or area under ROC curve;

[0232] When the task type is clustering type, the evaluation index is silhouette coefficient;

[0233] When the task type is anomaly detection type, the evaluation index is area under precision-recall curve or Matthew correlation coefficient.

[0234] In an embodiment of the present application, the candidate model is optimized by a hyperparameter optimization algorithm, specifically implemented by a particle swarm optimization algorithm, including the following steps:

[0235] Determine the initialization position range of each particle based on the hyperparameter search space of the candidate model;

[0236] Determine the number of particles based on the data volume of the target data subset and the calculation resource constraint;

[0237] Set the inertia weight, which can be linearly decreased based on the number of iterations or fixed as a preset empirical value;

[0238] Set the learning factor (cognitive factor) and (social factor), which is set based on experience or adjusted based on the analysis task type;

[0239] Randomly initialize the position vector and velocity vector of each particle, and the position vector represents a set of candidate hyperparameter combinations;

[0240] Calculate the fitness value of each particle based on the validation set of the target data subset, and the fitness function is set according to the analysis task type:

[0241] Prediction type task: fitness value = 1 / (1+root mean square error);

[0242] Classification type task: fitness value = F1 score;

[0243] Clustering type task: fitness value = silhouette coefficient;

[0244] Anomaly detection type task: fitness value = area under precision-recall curve or F1 score;

[0245] For each particle, compare the fitness value of its current position with the fitness value of its historical optimal position, and if the current position is better, update the historical optimal position as the global optimal position;

[0246] Update the individual optimal position and the global optimal position of each particle;

[0247] Update the particle velocity and position according to the following formula:

[0248]

[0249] wherein, is the velocity vector of the particle at the iteration number ; is the position vector of the particle at the iteration number , i.e. the hyperparameter combination; , is a learning factor for controlling the weight of individual experience and social cooperation; , is a random number; is the historical optimal position of the particle ; is the global optimal position of the entire particle swarm; is the inertia weight;

[0250] Iterative calculation is performed based on the particle number, the inertia weight, the learning factor and the velocity until a preset iteration number is reached or the difference between multiple consecutive fitness values is less than a first fitness threshold;

[0251] The particle position corresponding to the global optimal position is taken as the hyperparameter configuration of the candidate model.

[0252] The particle swarm optimization algorithm is used in this embodiment to search for optimal hyperparameters, which can converge to the global optimal solution faster and reduce the consumption of computing resources compared with traditional grid search or random search. Secondly, the linear decreasing strategy is used to balance the global search and local development capabilities, thereby improving the optimization efficiency. The weight of individual experience and social cooperation is adjusted according to the task type to adapt to the optimization requirements of different data characteristics. Special fitness functions are designed for different analysis tasks to ensure that the optimization target is consistent with the business requirements. Through the dual control of the iteration number and the fitness change threshold, over-optimization is avoided, and the model performance and the computing cost are balanced. The particle number is dynamically adjusted according to the data volume and the computing resources to avoid resource waste while ensuring the optimization effect, which is suitable for medical big data scenarios.

[0253] In a specific embodiment, multi-source heterogeneous data is obtained from a hospital medical service system, a medical insurance settlement system, a consumable supply chain system, and a government financial platform through an API interface. For unstructured medical record cost descriptions, entity recognition and relationship extraction are performed using a model to extract key information such as drug costs and examination item costs and convert them into structured data. All obtained data is aligned with data patterns and unified in format.

[0254] When determining the weight coefficients of each data source, data update frequency indicators, historical accuracy indicators, and data provider reputation level indicators are obtained. The weight coefficients of each data source are calculated through a formula, and based on this, weighted fusion is performed to generate multi-source heterogeneous data.

[0255] Isolation forest algorithm is used to detect and correct outliers, and clustering algorithm is used to identify and eliminate noise data. The data is processed for missing values to generate cleaned data. Based on the running analysis goals of hospital control of consumable costs and prediction of medical income, feature data sets are constructed from the cleaned data. Finally, random forest algorithm and Pearson correlation coefficient are used to screen features to generate target data sets.

[0256] The target data set is converted into a structured data set according to a star model. The time partition granularity is determined based on historical query frequency. If it is found that the quarterly query frequency has reached 40% in the past year and the data volume growth trend is stable, quarterly is used as the time partition granularity, and the data is divided into primary storage partitions.

[0257] According to the business department dimension, the cardiology department, respiratory department and other departments are coded, and the data records in the primary partition are divided into corresponding secondary sub-partitions; according to the data type dimension, the income, expenditure and other data are divided into tertiary sub-partitions. A partition metadata index table is established to record information such as partition time range, department code and data type code, and the data subsets are distributedly stored to the corresponding physical storage nodes.

[0258] The user issues a query request "predict the medical income of the cardiology department in the first quarter of 2025 and analyze the correlation between consumable use and income", and the system analyzes the analysis target parameters as time series prediction (prediction type), medical income (analysis index), and cardiology department in the first quarter of 2025 (business constraint condition).

[0259] Based on this, a candidate model set is selected from a preset model library, the time series prediction selects a long short-term memory network model, and the analysis of the correlation between consumable use and income selects a random forest model.

[0260] The candidate model is evaluated using the target data subset 2024-2025 cardiology-related sample data, the time series prediction uses root mean square error and mean absolute percentage error evaluation, and the consumable use and income correlation uses feature importance and correlation coefficient evaluation. If the evaluation index is greater than or equal to the preset index threshold, the time series prediction selects the long short-term memory network model; the analysis of the correlation between consumable use and income selects the random forest model, and optimization is not required.

[0261] The selected model is used to analyze the target data subset, and the long short-term memory network model predicts that the cardiology medical income in the first quarter of 2025 will be 8 million yuan, and the random forest model analysis shows that the correlation between the use of a certain type of high-value consumables and surgical income is 0.7.

[0262] Based on the analysis results, a report is generated using a template engine, including text descriptions, income prediction line graphs, consumable and income correlation bar charts, and other content, and is displayed to the user through a graphical user interface, supporting user data filtering, chart zooming, and other interactive operations.

[0263] Reference Figure 2 , flowchart of the data analysis method based on machine learning; the method includes the following functions:

[0264] HIS (hospital information system, manages diagnosis / treatment / charging, etc.), LIS (laboratory information system, manages laboratory data), PACS (imaging system, manages medical images), and other medical business systems as data sources, with the help of Kettle tools in the ETL platform, according to the rules set by the collection and scheduling, the data scattered in each system is extracted, and preliminary conversion is performed to adapt the data format, specification, etc. to the subsequent process.

[0265] Unify the obtained data, and according to the preset data modeling, sort out the data relationship and structure, so that the disordered data becomes regular and usable;

[0266] Load the data that completes the data modeling into the data warehouse that completes the physical modeling (designs the storage structure of the data warehouse).

[0267] According to the customer query request, match the relevant analysis model and the corresponding data;

[0268] Input the corresponding data into the analysis model to get the analysis result, wherein the analysis model can dig deep value of the data, predict future conditions, income trends and risk indicators, etc., and also can monitor key performance indicators.

[0269] Use visual large screens to visually display the analysis results (e.g., department daily income, income trend), making it easy for managers to grasp the hospital operation in real time.

[0270] Corresponding to the machine learning-based data analysis method in the above embodiments, Figure 3 This is a structural block diagram of a machine learning-based data analysis system provided in one embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 3 The machine learning-based data analysis system 20 includes: a data acquisition module 21, a data processing module 22, a data partitioning module 23, a data matching module 24, a model analysis module 25, and a data interaction module 26.

[0271] Among them, the data acquisition module 21 is used to acquire and integrate multi-source heterogeneous data sources of the target hospital. The multi-source heterogeneous data includes: medical business system data, medical insurance settlement data and consumable supply chain data.

[0272] Data processing module 22 is used to process multi-source heterogeneous data to obtain the target dataset;

[0273] The data partitioning module 23 is used to store the target dataset into multiple storage partitions based on a preset data model and partitioning rules, resulting in multiple data subsets. The partitioning rules support dividing the data according to at least one dimension, such as time range, business department, and data type, and the partitioning granularity is adjusted according to the historical query frequency.

[0274] Data matching module 24 is used to parse the filtering conditions in the user query request and match the target data subset from multiple data subsets;

[0275] The model analysis module 25 is used to select an analysis model from a preset model library based on the analysis target parameters and partition attributes of the target data subset in the query request, analyze the target data subset, and output the results.

[0276] The data interaction module 26 is used to generate corresponding reports based on the analysis results and interact with users through a graphical user interface.

[0277] See Figure 4 , Figure 4 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 4 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the aforementioned device embodiments, for example...Figure 3 The functions of the data acquisition module 21, the data processing module 22, the data partition module 23, the data matching module 24, the model analysis module 25 and the data interaction module 26 are shown.

[0278] It should be understood that, in the embodiments of the present application, the processor 301 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0279] The input device 302 can include a touchpad, a fingerprint acquisition sensor (for acquiring fingerprint information and direction information of a fingerprint of a user), a microphone, etc., and the output device 303 can include a display (LCD, etc.), a speaker, etc.

[0280] The memory 304 can include read-only memory and random access memory, and provide instructions and data for the processor 301. A part of the memory 304 can further include non-volatile random access memory. For example, the memory 304 can also store device type information.

[0281] In specific implementations, the processor 301, the input device 302 and the output device 303 described in the embodiments of the present application can perform the implementation manner described in any embodiment of the machine learning-based digital operation analysis method provided by the embodiments of the present application, and can also perform the implementation manner of the electronic device described in the embodiments of the present application, which will not be described here.

[0282] In another embodiment of the present application, a computer readable storage medium is provided, which stores a computer program. The computer program includes program instructions, which, when executed by a processor, implement all or part of the processes of the above-mentioned embodiment methods. The computer program can also instruct related hardware to complete the implementation. The computer program can be stored in a computer readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0283] The computer readable storage medium can be an internal storage unit of the electronic device of any of the preceding embodiments, such as a hard disk or a memory of the electronic device. The computer readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the electronic device. The computer readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0284] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0285] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic device and the units described above can refer to the corresponding processes in the above-mentioned method embodiments, which will not be described here.

[0286] In several embodiments provided in the present application, it should be understood that the disclosed electronic device and method can be implemented in other manners. For example, the embodiments of the apparatus described above are merely schematic; the division of the units is merely logical function division; an actual implementation can be divided into different units depending on actual conditions; or a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, or can be in electrical, mechanical or other forms.

[0287] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0288] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0289] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto; any skilled person in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be encompassed in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A machine learning based data analysis method, characterized by, The application relates to a method for processing multi-source heterogeneous data of a target hospital, and the method comprises the following steps: acquiring and fusing multi-source heterogeneous data of a target hospital, wherein the multi-source heterogeneous data comprises medical business system data, medical insurance settlement data and consumable supply chain data; processing the multi-source heterogeneous data to obtain a target data set; storing the target data set into a plurality of storage partitions based on a preset data model and partition rules to obtain a plurality of data subsets, wherein the partition rules support division of data according to at least one dimension of a time range, a business department and a data type, and the partition granularity is adjusted according to historical query frequency; analyzing filter conditions in a user query request to obtain a query time interval, a query department set and a query data type code; querying a partition metadata index table to screen partitions that meet the screening conditions to obtain screened partitions; determining loaded data in the plurality of data subsets according to the overlap degree of the time range of the screened partitions and the query time interval, wherein, when the overlap degree is less than a first overlap threshold, the partition is skipped; when the overlap degree is greater than or equal to the first overlap threshold and the overlap degree is less than a second overlap threshold, data overlapping the query time range and the partition time range is loaded; when the overlap degree is greater than or equal to the second overlap threshold, data of the partition is loaded; the loaded data is combined into a target data subset; the determination method of the first overlap threshold and the second overlap threshold comprises: based on historical query data, the relationship between data loading efficiency and query response time under different overlap degrees is analyzed, and an overlap degree-performance index prediction model is established; the shortest query response time and the smallest data loading resource consumption are taken as targets, and the optimal first overlap threshold and the second overlap threshold are calculated; an analysis model is selected from a preset model library according to an analysis target parameter in the query request and the partition attribute of the target data subset, the target data subset is analyzed and an analysis result is output; a corresponding report is generated based on the analysis result, and the report is interacted with a user through a graphical user interface; the selection of the analysis model from the preset model library according to the analysis target parameter in the query request and the partition attribute of the target data subset comprises: an analysis task type parameter in the query request is analyzed, a candidate model set is screened from the preset model library based on the analysis task type parameter and the key data characteristics of the target data subset; sample data of the target data subset is used to evaluate the performance of the candidate model, and evaluation indexes are set according to the task type; if any key evaluation index is lower than a preset threshold, the candidate model is optimized through a hyperparameter optimization algorithm, and the optimized model is taken as the final analysis model.

2. The machine learning based data analysis method of claim 1, wherein, The method for acquiring the multi-source heterogeneous data comprises the following steps: structured data and unstructured data of internal systems and external data platforms of a target hospital are acquired through API interfaces; an entity recognition and relationship extraction model is used to perform entity recognition and relationship extraction on the unstructured data to generate structured processing data; the structured data and the structured processing data are subjected to data mode alignment and format unification processing; the aligned data is subjected to weighted fusion based on weight coefficients of each data source of the target hospital to generate multi-source heterogeneous data.

3. The machine learning based data analysis method of claim 2, wherein, The method for determining the weight coefficient of each data source comprises: obtaining evaluation index data of each data source of a target hospital, the evaluation index data comprising a data update frequency index, a historical accuracy rate index and a data provider reputation level index; calculating the weight coefficient of each data source according to the evaluation index data and a preset first weight calculation formula.

4. The machine learning based data analysis method of claim 3, wherein, After calculating the weight coefficient of each data source according to the evaluation index data and the preset first weight calculation formula, the method further comprises: in response to the data update frequency index being less than a first update frequency threshold, reducing the weight coefficient of the data source by a first step size; in response to the data update frequency index being less than a second update frequency threshold and greater than or equal to the first update frequency threshold, reducing the weight coefficient of the data source by a second step size; wherein the first update frequency threshold is less than the second update frequency threshold, and the first step size is greater than the second step size.

5. The machine learning based data analysis method of claim 1, wherein, The method for storing the target data set into a plurality of storage partitions based on a preset data model and partition rules comprises: converting the target data set into a structured data set according to the preset data model; determining a time partition granularity based on historical query frequency, and dividing the structured data set into primary storage partitions according to a time range dimension; performing secondary sub-partition division on data records in the primary storage partitions according to a business department dimension; performing tertiary sub-partition division on data records in the secondary sub-partitions according to a data type dimension; establishing a partition metadata index table to record the time range, business department code and data type code of each partition; distributing the data subsets after the tertiary division to corresponding physical storage nodes in a distributed manner to generate the plurality of data subsets.

6. The machine learning based data analysis method of claim 5, wherein, The method for determining the time partition granularity based on historical query frequency comprises: counting the query frequency of different time intervals in historical query requests; selecting a daily, monthly or yearly partition granularity according to the mapping relationship between the query frequency and the time interval.

7. The machine learning based data analysis method of claim 6, wherein, The method for selecting a daily, monthly or yearly partition granularity according to the mapping relationship between the query frequency and the time interval comprises: when the time interval meets a first time interval condition and the corresponding query frequency is greater than a first query frequency threshold, adopting a daily partition granularity; when the time interval meets a second time interval condition and the corresponding query frequency is greater than a second query frequency threshold, adopting a monthly partition granularity; when the time interval meets a third time interval condition and the corresponding query frequency is greater than a third query frequency threshold, adopting a yearly partition granularity. 8.The machine learning based data analysis method of claim 1, wherein, The filtering condition comprises that the partition time range overlaps with the query time interval, the partition business department code belongs to the query department set, and the partition data type code matches the query data type code.

9. A machine learning based data analysis system, characterized by, The method, the system, comprising: a data acquisition module configured to acquire and fuse a plurality of source heterogeneous data of a target hospital, the plurality of source heterogeneous data comprising medical business system data, medical insurance settlement data and consumable supply chain data; a data processing module configured to process the plurality of source heterogeneous data to obtain a target data set; and The data partition module is configured to store the target data set into a plurality of storage partitions based on a preset data model and a partition rule, to obtain a plurality of data subsets; the partition rule supports dividing data according to at least one of a time range, a business department, and a data type, and a partition granularity is adjusted according to a historical query frequency; The data matching module is configured to analyze filter conditions in a user query request to obtain a query time interval, a query department set, and a query data type code; The query partition metadata index table is configured to filter partitions that meet the filter conditions to obtain filtered partitions; According to an overlap degree between a time range of the filtered partitions and the query time interval, loading data in the plurality of data subsets is determined, When the overlap degree is less than a first overlap threshold, the partition is skipped; When the overlap degree is greater than or equal to the first overlap threshold and less than a second overlap threshold, data overlapping the query time range and the partition time range is loaded; When the overlap degree is greater than or equal to the second overlap threshold, data of the partition is loaded; The loading data is combined into a target data subset; The determination method of the first overlap threshold and the second overlap threshold includes: Based on historical query data, a relationship between data loading efficiency and query response time under different overlap degrees is analyzed, and an overlap degree-performance index prediction model is established; With the shortest query response time and the smallest data loading resource consumption as targets, optimal first and second overlap thresholds are calculated; The model analysis module is configured to select an analysis model from a preset model library according to an analysis target parameter in the query request and a partition attribute of the target data subset, analyze the target data subset, and output a result; The selection of the analysis model from the preset model library according to the analysis target parameter in the query request and the partition attribute of the target data subset includes: analyzing a task type parameter in the query request, the parameter including a task type, an analysis index, and a business constraint condition; Based on the task type parameter and key data characteristics of the target data subset, a candidate model set is selected from the preset model library; the key data characteristics include at least one of a data volume, a feature dimension, a time sequence characteristic, and a label distribution; Performance of the candidate model is evaluated using sample data of the target data subset, and an evaluation index is set according to the task type; If any key evaluation index is lower than a preset threshold, the candidate model is optimized through a hyperparameter optimization algorithm, and the optimized model is used as a final analysis model; The data interaction module is configured to generate a corresponding report based on the analysis result and interact with a user through a graphical user interface.

Images