Data traffic control method, system and device, equipment and storage medium

Abstract

The invention discloses a data traffic control method, system, device and equipment and a storage medium, and the data traffic control method comprises the steps: obtaining target medical data traffic information in a current time period; the target medical data traffic information is input into a target prediction model, a target traffic prediction result in a future time period is obtained, and the target prediction model is obtained through training according to historical medical data traffic information; and adjusting task processing resources for processing the medical data based on the target traffic prediction result and the traffic threshold corresponding to the current time period. The method can play a role in improving the efficiency of adjusting the data processing resources.

Description

Technical Field

[0001] The present application belongs to the field of resource scheduling technology, and in particular relates to a data flow control method, system, device, equipment and storage medium. Background Art

[0002] Real-time processing technology plays a vital role in the healthcare sector. It can significantly help medical institutions and related organizations make quick decisions, effectively improve the quality of patient care, and achieve efficient resource management. Cloud-native architecture, with its high scalability, flexibility, and ease of maintenance, has become an ideal choice for processing massive amounts of data and complex workflows in the medical industry. The close integration of real-time processing technology and cloud-native technology has become an important development trend in the field of modern data processing, especially in application scenarios that process large-scale real-time data. This combination gives the system higher processing efficiency and flexibility, enabling it to easily handle real-time data streams from various sources and quickly respond to data changes.

[0003] When real-time processing jobs are initiated, the system often adopts a distributed architecture, continuously processing the influx of data through multiple tasks running in parallel. However, in the actual application scenarios of big health, the generation of medical data is not continuous and stable. Business data traffic exhibits a clear tidal fluctuation characteristic, with data peaks during the day and lows at night, or even no data generated at all. To ensure business stability, traditional practices often pre-determine peak data processing capacity based on technical experience, which often leads to a large amount of idle and wasted resources during non-peak hours. To cope with such traffic fluctuations, manual intervention is generally used to adjust the available data processing resources for each time period. However, this method of adjusting the available data processing resources for different time periods is inefficient. Summary of the Invention

[0004] The embodiments of the present application provide an implementation solution that is different from the prior art to solve the technical problem of low adjustment efficiency of data processing resources in the related art.

[0005] In the first aspect, the present application provides a data flow control method, including: obtaining target medical data flow information within a current time period; inputting the target medical data flow information into a target prediction model to obtain a target flow prediction result within a future time period, wherein the target prediction model is trained based on historical medical data flow information; and adjusting the task processing resources used to process medical data based on the flow threshold corresponding to the target flow prediction result and the current time period.

[0006] In the second aspect, the present application provides a data flow control system, including: a flow prediction module and a job management module; the flow prediction module is used to obtain target medical data flow information in a current time period; the target medical data flow information is input into a target prediction model to obtain a target flow prediction result in a future time period, wherein the target prediction model is trained based on historical medical data flow information; the job management module is used to adjust the task processing resources used to process medical data based on the flow threshold corresponding to the target flow prediction result and the current time period.

[0007] In a third aspect, the present application provides a data flow control device, comprising:

[0008] An acquisition unit, used to acquire target medical data flow information within a current time period;

[0009] An input unit, configured to input the target medical data flow information into a target prediction model to obtain a target flow prediction result for a future time period, wherein the target prediction model is trained based on historical medical data flow information;

[0010] An adjustment unit is used to adjust the task processing resources used to process medical data based on the target flow prediction result and the flow threshold corresponding to the current time period.

[0011] In a fourth aspect, the present application provides an electronic device, comprising:

[0012] processor; and

[0013] a memory for storing executable instructions of the processor;

[0014] The processor is configured to execute any method in the first aspect or any possible implementation manner of the first aspect by executing the executable instructions.

[0015] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any method in the first aspect or any possible implementation manner of the first aspect.

[0016] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in the first aspect or any of the possible implementations of the first aspect.

[0017] The present application provides a method for obtaining target medical data flow information within a current time period; inputting the target medical data flow information into a target prediction model to obtain a target flow prediction result within a future time period, wherein the target prediction model is trained based on historical medical data flow information; and adjusting task processing resources for processing medical data based on the target flow prediction result and the flow threshold corresponding to the current time period. The target flow prediction result within a future time period can be obtained based on the target prediction model trained based on historical medical data flow information and the target medical data flow information within the current time period, and the task processing resources for processing medical data can be adjusted based on the target flow prediction result and the flow threshold corresponding to the current time period, thereby realizing the function of automatically adjusting task processing resources for processing medical data and improving the efficiency of resource adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0019] Figure 1a A schematic diagram of the structure of a data flow control system provided in one embodiment of the present application;

[0020] Figure 1b A schematic diagram of the interaction between the traffic prediction module and the job management module is provided for an embodiment of the present application;

[0021] Figure 2a A flow chart of a data flow control method provided in one embodiment of the present application;

[0022] Figure 2b Another flowchart of a data flow control method according to an embodiment of the present application is provided;

[0023] Figure 3 A schematic diagram of the structure of a data flow control device provided in one embodiment of the present application;

[0024] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to be used to explain the present application, but should not be understood as limiting the present application.

[0026] The terms "first" and "second" in the specification, claims and drawings of the embodiments of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] First, some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0028] Cloud-native architecture is an architectural model based on cloud computing concepts that emphasizes designing, building, and running applications in a cloud environment. It leverages the characteristics and advantages of cloud computing, using technologies such as containerization, microservices, automated management, and continuous delivery to achieve flexible, scalable, and efficient application operations.

[0029] The SARIMA (Seasonal Autoregressive Integrated Moving Average) model is a time series forecasting model particularly suitable for data series with seasonal variations. It is an extension of the ARIMA (Autoregressive Integrated Moving Average) model, specifically designed for processing time series data with seasonal variations. The SARIMA model combines the characteristics of autoregression (AR), differencing (I), and moving average (MA), as well as seasonal autoregression (SAR), seasonal differencing (SI), and seasonal moving average (SMA), enabling more accurate capture and forecasting of data series with seasonal patterns. The model uses differencing to make non-stationary data stationary and leverages seasonal factors to improve forecast accuracy. The selection of parameters in the SARIMA model is crucial, including non-seasonal parameters (such as the autoregressive order p, differencing order d, and moving average order q) and seasonal parameters (such as the seasonal autoregressive order P, seasonal differencing order D, seasonal moving average order Q, and seasonal period m). The SARIMA model can be used to model and forecast data with significant seasonal fluctuations, such as patient visits in the healthcare industry and sales in the e-commerce industry.

[0030] The ACF (Autocorrelation Function) plot is a commonly used tool in time series analysis, used to describe the correlation between data values ​​at different time points in the same time series. The ACF plot shows the strength of the correlation between the data value at the current time point and the data values ​​at previous time points. It is often used to diagnose the stationarity of time series data, determine parameters in ARIMA (or SARIMA) models, and assess whether the model's residuals are white noise. In an ACF plot, the horizontal axis represents the time lag (lag), which is the time interval between the current time point and a previous time point. The vertical axis represents the autocorrelation coefficient (autocorrelation coefficient), which ranges from -1 to 1. An autocorrelation coefficient close to 1 indicates a strong positive correlation, close to -1 indicates a strong negative correlation, and close to 0 indicates no correlation.

[0031] The PACF plot (Partial Auto-Correlation Function) is also a commonly used tool in time series analysis. It complements the ACF plot (autocorrelation function plot) and is used to further analyze the correlation structure in time series data. The PACF plot focuses on the direct correlation between the data value at the current time point and the data values ​​at previous time points after excluding the influence of intermediate variables. In the PACF plot, the horizontal axis also represents the time lag (lag), that is, the time interval between the current time point and a certain time point in the past; the vertical axis represents the partial autocorrelation coefficient (Partial Autocorrelation Coefficient), and its value also ranges between -1 and 1. A partial autocorrelation coefficient close to 1 indicates a high positive correlation, close to -1 indicates a high negative correlation, and close to 0 indicates no direct correlation.

[0032] Real-time processing technology plays a vital role in the healthcare sector. It can significantly help medical institutions and related organizations make quick decisions, effectively improve the quality of patient care, and achieve efficient resource management. Cloud-native architecture, with its high scalability, flexibility, and ease of maintenance, has become an ideal choice for processing massive amounts of data and complex workflows in the medical industry. The close integration of real-time processing technology and cloud-native technology has become an important development trend in the field of modern data processing, especially in application scenarios that process large-scale real-time data. This combination gives the system higher processing efficiency and flexibility, enabling it to easily handle real-time data streams from various sources and quickly respond to data changes.

[0033] When real-time processing jobs are initiated, the system often adopts a distributed architecture, continuously processing the influx of data through multiple tasks running in parallel. However, in the actual application scenarios of big health, the generation of medical data is not continuous and stable. Business data traffic exhibits a clear tidal fluctuation characteristic, with data peaks during the day and lows at night, or even no data generated at all. To ensure business stability, traditional practices often pre-determine peak data processing capacity based on technical experience, which often leads to a large amount of idle and wasted resources during non-peak hours. To cope with such traffic fluctuations, manual intervention is generally used to adjust the available data processing resources for each time period. However, this method of adjusting the available data processing resources for different time periods is inefficient.

[0034] In order to effectively solve the above problems, this application proposes a data flow control method to achieve refined dynamic management of task processing resources. Specifically, this method can integrate real-time data and historical data, and use advanced algorithms to predict data flow trends in the future; then, based on the prediction results and the current resource threshold, accurately calculate the resource parameters that need to be adjusted. The SARIMA algorithm is introduced, which can deeply explore the potential correlation between historical flow and real-time flow, accurately grasp the seasonality, trend and non-stationary characteristics of clinical data, and then more reasonably predict the time band of business data volume. Based on the prediction results, the system can flexibly adjust resource allocation, which not only reduces the cost of resource use, but also improves the maintainability of resource management and the overall performance of the system.

[0035] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0036] Figure 1a A schematic structural diagram of a data flow control system provided by an exemplary embodiment of the present application, the system includes: a flow prediction module and a job management module;

[0037] The traffic prediction module is used to obtain target medical data traffic information in a current time period; input the target medical data traffic information into a target prediction model to obtain a target traffic prediction result in a future time period, wherein the target prediction model is trained based on historical medical data traffic information;

[0038] Optionally, the aforementioned future time period may take the current moment as the starting moment, and the duration of the future time period may be a preset duration.

[0039] Optionally, the aforementioned future time period may take the start collection time of the current day as the starting time, and the duration of the future time period may be the total duration of the preset duration and the duration of the current time period.

[0040] Optionally, when collecting medical data flow information, the collection interval can be set by relevant personnel. When collecting medical data flow information, it can be segmented according to certain statistical time periods. For example, the medical data flow information generated each day can be a segment of data. 0:00 each day is the starting time for collection of the corresponding data segment.

[0041] Optionally, the current time period may be the start collection time of the day, for example, midnight of the day. The target medical data flow information in the current time period is the medical data flow information collected in the current time period.

[0042] The aforementioned target medical data flow information may include multiple pieces of medical data flow information. Optionally, each piece of medical data flow information includes at least one of the following: the time of collection of the medical data flow information, a time period characteristic of the medical data flow information, and a flow value of the medical data flow information. Time period characteristics may include hourly, morning and afternoon, weekday, and holiday time period characteristics, which are used to capture periodic changes in data over different time periods. The flow value may refer to the flow volume within each statistical period, typically measured in MiB / h (megabytes per hour). The statistical period may be longer than the collection period but shorter than the aforementioned statistical period.

[0043] Optionally, the aforementioned preset duration can be determined according to a parameter seasonal cycle in the target prediction model. For example, the product of the parameter value of the seasonal cycle and the preset duration can be 24, the number of hours in a day.

[0044] In some optional embodiments of the present application, the target flow prediction results in the future time period and the target medical data flow information in the current time period can be presented in the form of a flow waveform graph to reflect the flow change trend over time.

[0045] The job management module is used to adjust the task processing resources for processing medical data based on the target traffic prediction result and the traffic threshold corresponding to the current time period.

[0046] In some optional embodiments of the present application, adjusting the task processing resources for processing medical data based on the target traffic prediction result and the traffic threshold corresponding to the current time period includes:

[0047] When the target flow prediction result indicates that the total flow value in the future time period is greater than the flow threshold corresponding to the current time period, and the difference between the total flow value and the flow threshold is greater than a preset threshold, controlling the expansion of task processing resources for processing medical data;

[0048] When the target flow prediction result indicates that the total flow value in the future time period is not greater than the flow threshold corresponding to the current time period, and the absolute value of the difference between the total flow value and the flow threshold is greater than the preset threshold, the control is used to reduce the task processing resources used to process medical data.

[0049] Optionally, the aforementioned preset threshold can be set by relevant personnel, for example, it can be set to 50 MiB / s.

[0050] In some optional embodiments of the present application, controlling the expansion of task processing resources for processing medical data includes: applying to the cloud native platform to add a task manager for processing medical data, and registering the added task manager with the job management module.

[0051] In some optional embodiments of the present application, controlling the reduction of task processing resources used to process medical data includes: controlling the task manager to be reduced to stop processing data, controlling the task manager to be reduced to go offline, and applying to the cloud native platform to clear the task manager to be reduced.

[0052] In some optional embodiments of the present application, the job management module includes: an event monitoring module and a task control module;

[0053] The event monitoring module is configured to monitor the target flow prediction result from the flow prediction module, and determine whether it is necessary to adjust the task processing resources for processing medical data based on the target flow prediction result and the flow threshold corresponding to the current time period, and if so, send adjustment notification information to the task control module;

[0054] The task control module is used to adjust the task processing resources used to process the medical data based on the adjustment notification information.

[0055] In some optional embodiments of the present application, the task control module includes a control unit and at least one task manager; the control unit is used to adjust the working status of any one or more task managers among the at least one task manager based on the adjustment notification information to adjust the task processing resources used to process medical data.

[0056] In some optional embodiments of the present application, the aforementioned adjustment notification information may include prompt information for prompting adjustment of task processing resources for processing medical data, and the number of task managers to be adjusted;

[0057] Specifically, when the target flow prediction result indicates that the total flow value in the future time period is greater than the flow threshold corresponding to the current time period, and the difference between the total flow value and the flow threshold is greater than a preset threshold, the adjustment notification information may include prompt information for prompting the expansion of task processing resources for processing medical data, as well as the number of task managers to be expanded;

[0058] When the target flow prediction result indicates that the total flow value in the future time period is not greater than the flow threshold corresponding to the current time period, and the absolute value of the difference between the total flow value and the flow threshold is greater than a preset threshold, the adjustment notification information may include prompt information for prompting to reduce the task processing resources used to process the medical data, and the number of task managers to be reduced;

[0059] Optionally, the number of task managers to be expanded is determined based on the numerical interval of the difference between the total flow value and the flow threshold. The number of task managers to be reduced is determined based on the numerical interval of the absolute value of the difference between the total flow value and the flow threshold.

[0060] Any one or more of the aforementioned task managers are task managers that need to be adjusted.

[0061] In some optional embodiments of the present application, when the target traffic prediction result indicates that the total traffic value in the future time period is greater than the traffic threshold corresponding to the current time period, and the difference between the total traffic value and the traffic threshold is greater than the preset threshold, the aforementioned adjustment of the working status of any one or more task managers in the at least one task manager refers to the control unit marking any one or more task managers in the at least one task manager as Paused Receiving state, and then each task manager to be adjusted notifies its respective source operator to no longer receive new data, and when the control unit detects that the data on the task manager to be adjusted has been completely processed, the task manager is switched to offline mode. After each task manager to be adjusted is switched to offline mode, the control unit uses the API provided by the cloud native platform (such as Kubernetes) to adjust the resource application of the stopped task manager.

[0062] In some optional embodiments of the present application, when the target traffic prediction result indicates that the total traffic value in the future time period is greater than the traffic threshold corresponding to the current time period, and the absolute value of the difference between the total traffic value and the traffic threshold is greater than the preset threshold, the control unit can call the API provided by the cloud native platform according to the configured parameters to extend the task manager, and register the task manager with the job manager to complete the expansion of resources.

[0063] After the control unit has finished adjusting the resources, the task management module sends the resource adjustment result to the result queue in the form of a message.

[0064] Further, see Figure 1b As shown, Figure 1b A schematic diagram of the interaction between the traffic prediction module and the job management module provided in an embodiment of the present application. The traffic prediction module inputs the target traffic prediction results into a message queue, from which the event monitoring module retrieves them. After adjusting the task processing resources used to process medical data, the task control module sends the resource adjustment results to the message queue, from which the traffic prediction module retrieves them for easy recording.

[0065] This solution reduces the coupling between the two modules. Even if the traffic prediction model is not working, the job management module can still function normally, ensuring system stability and reliability. It can dynamically scale up and down real-time tasks without interrupting business operations, thus ensuring stable system operation and business continuity.

[0066] The execution principles and interaction processes of each component unit in the embodiment of this system can be found in the description of the following method embodiments.

[0067] Figure 2a A flow chart of a data flow control method provided in an exemplary embodiment of the present application is provided. The execution subject of the method may be a server. The method includes at least the following steps S201-S203:

[0068] S201, obtaining target medical data flow information within the current time period;

[0069] Optionally, when collecting medical data flow information, the collection interval can be set by relevant personnel. When collecting medical data flow information, it can be segmented according to certain statistical time periods. For example, the medical data flow information generated each day can be a segment of data. 0:00 each day is the starting time for collection of the corresponding data segment.

[0070] Optionally, the current time period may be the start collection time of the day, for example, midnight of the day. The target medical data flow information in the current time period is the medical data flow information collected in the current time period.

[0071] The aforementioned target medical data flow information may include multiple pieces of medical data flow information. Optionally, each piece of medical data flow information includes at least one of the following: the time of collection of the medical data flow information, a time period characteristic of the medical data flow information, and a flow value of the medical data flow information. Time period characteristics may include hourly, morning and afternoon, weekday, and holiday time period characteristics, which are used to capture periodic changes in data over different time periods. The flow value may refer to the flow volume within each statistical period, typically measured in MiB / h (megabytes per hour). The statistical period may be longer than the collection period but shorter than the aforementioned statistical period.

[0072] S202: Input the target medical data flow information into a target prediction model to obtain a target flow prediction result for a future time period, wherein the target prediction model is trained based on historical medical data flow information;

[0073] Optionally, the aforementioned future time period may take the current moment as the starting moment, and the duration of the future time period may be a preset duration.

[0074] Optionally, the aforementioned future time period may take the start collection time of the current day as the starting time, and the duration of the future time period may be the total duration of the preset duration and the duration of the current time period.

[0075] Optionally, the aforementioned preset duration can be determined according to a parameter seasonal cycle in the target prediction model. For example, the product of the parameter value of the seasonal cycle and the preset duration can be 24, the number of hours in a day.

[0076] In some optional embodiments of the present application, the target flow prediction results in the future time period and the target medical data flow information in the current time period can be presented in the form of a flow waveform graph to reflect the flow change trend over time.

[0077] In some optional embodiments of the present application, the method further includes the following S1-S2:

[0078] S1. Determine a sample data set for training the target prediction model based on the historical medical data flow information;

[0079] Optionally, the historical medical data flow information may include multiple medical data flow records within a preset historical time period, each of which includes at least one of the following: the time of collection of the medical data flow information, a time period characteristic of the medical data flow information, and a flow value of the medical data flow information. Time period characteristics may include hourly, morning and afternoon, weekday, and holiday time period characteristics, which are used to capture periodic changes in data over different time periods. The flow value may refer to the flow volume within each statistical period, typically measured in MiB / h (megabytes per hour). The statistical period may be longer than the collection period but shorter than the aforementioned statistical time period.

[0080] Optionally, the aforementioned historical time period may be the past week, and the aforementioned statistical period may be one hour.

[0081] In some optional embodiments of the present application, after collecting historical medical data flow information and target medical data flow information, the collected data needs to be preprocessed, such as data cleaning, outlier processing, etc., to ensure data quality.

[0082] In some optional embodiments of the present application, in the aforementioned S1, determining the sample data set for training the target prediction model based on the historical medical data flow information includes the following steps S11-S12:

[0083] S11. Acquire multiple data sorting rules, wherein the multiple data sorting rules include sorting data by time period, sorting data by data change trends in the same season of different years, and sorting data by external factors, wherein the external factors include weather information, whether there is an epidemic, and whether it is a period of traffic disease outbreak;

[0084] Optionally, the aforementioned time period may include any one or more of the following: hours, mornings and afternoons, weeks, and holidays.

[0085] S12. For each data sorting rule among the multiple data sorting rules, sort the historical medical data flow information to obtain a sorting result corresponding to the data sorting rule, and then obtain a sample data set for training the target prediction model.

[0086] Specifically, each data sorting rule corresponds to a sorting result of historical medical data flow information.

[0087] S2. Obtain the target prediction model based on the training of the sample data set.

[0088] In some optional embodiments of the present application, in the aforementioned S2, the training based on the sample data set to obtain the target prediction model includes the following steps S21-S22:

[0089] S21, grouping the sample data set to obtain multiple groups of sample data, wherein each group of sample data includes partial data sorted based on various data sorting rules;

[0090] That is, each set of sample data includes the data sorted according to the plurality of data sorting rules. Optionally, the sample data in different sample data sets are different.

[0091] S22. Obtain the target prediction model based on the training of the multiple groups of sample data.

[0092] In some optional embodiments of the present application, in S22, the training based on the multiple groups of sample data to obtain the target prediction model includes the following steps S221-S223:

[0093] S221: For each set of sample data in the plurality of sets of sample data, train an initial prediction model using the sample data, thereby obtaining a plurality of initial prediction models corresponding to the plurality of sets of sample data;

[0094] In some optional embodiments of the present application, the target prediction model is a seasonal autoregressive moving average SARIMA model.

[0095] Optionally, the initial prediction model in this application is an initial SARIMA model, which combines the characteristics of autoregression (AR), differencing (I), moving average (MA), and seasonal autoregression (SAR), seasonal differencing (SI), and seasonal moving average (SMA). The SARIMA formula can be shown as follows:

[0096] SARIMA(p,d,q)×(P,D,Q)m

[0097] Specifically, the SARIMA model includes two types of parameters: non-seasonal parameters and seasonal parameters. The non-seasonal parameters include the differencing order d, the autoregressive order p, and the moving average order q. The seasonal parameters include the seasonal period m, the seasonal differencing order D, the seasonal autoregressive order P, and the seasonal moving average order Q.

[0098] In some optional embodiments of the present application, after an initial prediction model is obtained by training each group of sample data in the multiple groups of sample data using the sample data, for the differential order d in the initial prediction model, the ACF (Autocorrelation Function) graph is observed to determine whether the data needs to be differentiated to make it stable. If it is stable, the initial prediction model is no longer trained. If it is not stable, the initial prediction model continues to be trained.

[0099] The autoregressive order p in the initial prediction model is determined by the PACF (Partial Autocorrelation Function) graph. If the PACF graph shows a sharp drop after a certain point, the initial prediction model is continued to be trained. If not, the initial prediction model is no longer trained.

[0100] The moving average order q in the initial prediction model is determined by the ACF graph. If the ACF graph shows a sharp drop after a certain point, the initial prediction model continues to be trained. Otherwise, the initial prediction model is no longer trained.

[0101] For the seasonal period m in this initial forecast model, determine the seasonal period based on the frequency of the data. If the data can be recorded hourly and there is daily seasonal variation, m can be set to 24 or 12, etc.

[0102] The seasonal difference order D in the initial forecast model is determined by observing the seasonal ACF plot. If the seasonal ACF plot shows a clear seasonal pattern, the initial forecast model continues to be trained. If not, the initial forecast model is no longer trained.

[0103] The seasonal autoregressive order P in the initial forecast model is determined by the seasonal PACF plot. If the seasonal ACF plot shows a clear seasonal pattern, the initial forecast model is continued to be trained. If not, the initial forecast model is no longer trained.

[0104] The seasonal moving average order Q in the initial forecast model is determined by the seasonal ACF plot. If the seasonal ACF plot shows a clear seasonal pattern, the initial forecast model continues to be trained. If not, the initial forecast model is no longer trained.

[0105] Specifically, the aforementioned ACF graphs and PACF graphs can be determined based on historical medical data flow information. The values ​​of d, p, and q are 0, 1, or between 0 and 1.

[0106] In some optional embodiments of the present application, during the process of training the initial prediction model using the sample data, a loss value between the traffic prediction result output by the initial prediction model and the actual traffic result may be calculated, and the parameters of the initial prediction model may be adjusted based on the principle of minimizing the loss value. The loss value may be an MSE (mean square error) value, an MAE (mean absolute error) value, or the like.

[0107] Specifically, when the loss value is less than a preset value or the number of iterations reaches a preset number, it is determined that the training of the initial prediction model based on the current group of sample data is completed. Each group of sample data corresponds to a trained initial prediction model.

[0108] S222: Score the performance of each initial prediction model in the multiple initial prediction models to obtain a scoring result for each initial prediction model;

[0109] When scoring each initial prediction model, part of the sample data in the sample data set can be used as a test set to score the initial prediction model. Specifically, the scoring indicators may include MSE (mean square error) or MAE (mean absolute error). Specifically, the smaller the MSE value, the higher the score of the corresponding initial prediction model, and the smaller the MAEMSE value, the higher the score of the corresponding initial prediction model.

[0110] S223. Determine the target prediction model according to the scoring results of the initial prediction models.

[0111] Specifically, determining the target prediction model according to the scoring results of each initial prediction model includes:

[0112] Among the scoring results of the initial prediction models, the initial prediction model with the highest score is used as the target prediction model.

[0113] S203: Adjust the task processing resources for processing medical data based on the target traffic prediction result and the traffic threshold corresponding to the current time period.

[0114] Optionally, the traffic threshold corresponding to the aforementioned current time period can be set by relevant personnel.

[0115] In some optional embodiments of the present application, in the aforementioned S203, adjusting the task processing resources for processing medical data based on the target traffic prediction result and the traffic threshold corresponding to the current time period includes:

[0116] When the target flow prediction result indicates that the total flow value in the future time period is greater than the flow threshold corresponding to the current time period, and the difference between the total flow value and the flow threshold is greater than a preset threshold, controlling the expansion of task processing resources for processing medical data;

[0117] When the target flow prediction result indicates that the total flow value in the future time period is not greater than the flow threshold corresponding to the current time period, and the absolute value of the difference between the total flow value and the flow threshold is greater than the preset threshold, the control is used to reduce the task processing resources used to process medical data.

[0118] Optionally, the aforementioned preset threshold can be set by relevant personnel, for example, it can be set to 50 MiB / s.

[0119] In some optional embodiments of the present application, controlling the expansion of task processing resources for processing medical data includes: applying to the cloud native platform to add a task manager for processing medical data, and registering the added task manager locally.

[0120] In some optional embodiments of the present application, controlling the reduction of task processing resources used to process medical data includes: controlling the task managers to be reduced to stop processing data, controlling the task managers to be reduced to go offline, and applying to the cloud native platform to clear some task managers used to process medical data.

[0121] Further, see Figure 2b As shown, Figure 2b A flow chart of a data flow control method provided in this application. When the job is started according to the instructions of relevant personnel or the preset start time, the flow prediction module starts working and transmits the target flow prediction result to the job management module. The job management module adjusts the task processing resources used to process medical data and transmits the resource adjustment result to the flow prediction module.

[0122] The solution of this application combines the characteristics of medical services and comprehensively uses historical data and real-time data to predict changes in business trends within a certain period of time in the future. It can continuously detect the operating status and traffic changes of the data processing system. According to the actual operating conditions, the prediction model parameters are adjusted or the resource adjustment strategy is optimized to improve system performance, resource utilization and responsiveness, reduce operation and maintenance costs, and improve the scalability and stability of the system. It can effectively cope with the volatility of real-time medical data traffic in medical institutions and improve the stability and resource utilization of the data processing system. By reasonably setting thresholds, unnecessary costs and efficiency losses caused by frequent resource adjustments can be effectively avoided, thereby ensuring the quality and efficiency of services.

[0123] The solution in this application is applicable to medical scenarios. For example, if a medical institution needs to process real-time patient data, data traffic peaks during the day and decreases significantly at night. To ensure the stability and efficiency of the data processing system, the automatic scaling method in this application is required.

[0124] Figure 3 A schematic diagram of a data flow control device provided as an exemplary embodiment of the present application; wherein the device includes:

[0125] An acquisition unit 31 is configured to acquire target medical data flow information within a current time period;

[0126] An input unit 32 is configured to input the target medical data flow information into a target prediction model to obtain a target flow prediction result for a future time period, wherein the target prediction model is trained based on historical medical data flow information;

[0127] The adjusting unit 33 is configured to adjust the task processing resources for processing medical data based on the target flow prediction result and the flow threshold corresponding to the current time period.

[0128] Optionally, when the aforementioned apparatus is used to adjust the task processing resources for processing medical data based on the target flow prediction result and the flow threshold corresponding to the current time period, it is specifically used to:

[0129] When the target flow prediction result indicates that the total flow value in the future time period is greater than the flow threshold corresponding to the current time period, and the difference between the total flow value and the flow threshold is greater than a preset threshold, controlling the expansion of task processing resources for processing medical data;

[0130] When the target flow prediction result indicates that the total flow value in the future time period is not greater than the flow threshold corresponding to the current time period, and the absolute value of the difference between the total flow value and the flow threshold is greater than the preset threshold, the control is used to reduce the task processing resources used to process medical data.

[0131] Optionally, the aforementioned device is further used for:

[0132] Determining a sample data set for training the target prediction model based on the historical medical data flow information;

[0133] The target prediction model is obtained by training based on the sample data set.

[0134] Optionally, when the aforementioned apparatus is used to determine a sample data set for training the target prediction model based on the historical medical data flow information, it is specifically used to:

[0135] Acquire multiple data sorting rules, including sorting data by time period, sorting data by data change trends in the same season of different years, and sorting data by external factors, wherein the external factors include weather information, whether there is an epidemic, and whether it is a traffic disease outbreak period;

[0136] According to each data sorting rule among the multiple data sorting rules, the historical medical data flow information is sorted to obtain a sorting result corresponding to the data sorting rule, and then a sample data set for training the target prediction model is obtained.

[0137] Optionally, when the aforementioned apparatus is used to train the target prediction model based on the sample data set, it is specifically used to:

[0138] Grouping the sample data set to obtain multiple groups of sample data, wherein each group of sample data includes part of the data sorted based on each data sorting rule;

[0139] The target prediction model is obtained by training based on the multiple groups of sample data.

[0140] Optionally, when the aforementioned apparatus is used to train the target prediction model based on the multiple groups of sample data, it is specifically used to:

[0141] For each set of sample data in the plurality of sets of sample data, an initial prediction model is obtained by training the sample data, thereby obtaining a plurality of initial prediction models corresponding to the plurality of sets of sample data;

[0142] Scoring the performance of each initial prediction model in the multiple initial prediction models to obtain a scoring result for each initial prediction model;

[0143] The target prediction model is determined according to the scoring results of the initial prediction models.

[0144] Optionally, the target prediction model is a seasonal autoregressive moving average SARIMA model.

[0145] It should be understood that the device embodiments and the method embodiments may correspond to each other, and similar descriptions may refer to the method embodiments. To avoid repetition, they will not be described in detail here. Specifically, the device can perform the above-mentioned method embodiments, and the aforementioned and other operations and / or functions of each module in the device are the corresponding processes in each method in the above-mentioned method embodiments, which will not be described in detail here for the sake of brevity.

[0146] The apparatus of the embodiment of the present application is described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, or can be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method embodiment in conjunction with its hardware.

[0147] Figure 3 : is a schematic block diagram of an electronic device provided in an embodiment of the present application, and the electronic device may include:

[0148] The memory 301 and the processor 302 are configured to store computer programs and transmit the program code to the processor 302. In other words, the processor 302 can call and run the computer program from the memory 301 to implement the method in the embodiment of the present application.

[0149] For example, the processor 302 may be configured to execute the above method embodiments according to instructions in the computer program.

[0150] In some embodiments of the present application, the processor 302 may include but is not limited to:

[0151] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0152] In some embodiments of the present application, the memory 301 includes but is not limited to:

[0153] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0154] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 301 and executed by the processor 302 to implement the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0155] like Figure 3 As shown, the electronic device may further include:

[0156] The transceiver 303 may be connected to the processor 302 or the memory 301 .

[0157] The processor 302 may control the transceiver 303 to communicate with other devices. Specifically, the processor 302 may send information or data to other devices or receive information or data sent by other devices. The transceiver 303 may include a transmitter and a receiver. The transceiver 303 may further include one or more antennas.

[0158] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0159] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, enables the computer to perform the method of the above-mentioned method embodiment. In other words, the present application also provides a computer program product containing instructions, which, when executed by a computer, enables the computer to perform the method of the above-mentioned method embodiment.

[0160] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0161] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0163] Modules described as separate components may or may not be physically separate, and components displayed as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected based on actual needs to achieve the purpose of the present embodiment. For example, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module.

[0164] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A data flow control method, characterized in that: include: Obtain target medical data flow information within the current time period; Inputting the target medical data flow information into a target prediction model to obtain a target flow prediction result in a future time period, wherein the target prediction model is trained based on historical medical data flow information; The task processing resources for processing medical data are adjusted based on the target traffic prediction result and the traffic threshold corresponding to the current time period.

2. The method according to claim 1, characterized in that The adjusting the task processing resources for processing medical data based on the target flow prediction result and the flow threshold value corresponding to the current time period includes: When the target flow prediction result indicates that the total flow value in the future time period is greater than the flow threshold corresponding to the current time period, and the difference between the total flow value and the flow threshold is greater than a preset threshold, control the expansion of task processing resources for processing medical data; When the target flow prediction result indicates that the total flow value in the future time period is not greater than the flow threshold corresponding to the current time period, and the absolute value of the difference between the total flow value and the flow threshold is greater than a preset threshold, the control is performed to reduce the task processing resources used to process medical data.

3. The method according to claim 1, characterized in that The method further comprises: Determining a sample data set for training the target prediction model according to the historical medical data flow information; The target prediction model is obtained by training based on the sample data set.

4. The method according to claim 3, characterized in that The step of determining the sample data set for training the target prediction model according to the historical medical data flow information includes: Acquire multiple data sorting rules, wherein the multiple data sorting rules include: sorting data according to time period, sorting data according to data change trends in the same season of different years, and sorting data according to external factors, wherein the external factors include: weather information, whether there is an epidemic, and whether it is a traffic disease outbreak period; According to each data sorting rule among the multiple data sorting rules, the historical medical data flow information is sorted to obtain a sorting result corresponding to the data sorting rule, and then a sample data set for training the target prediction model is obtained.

5. The method according to claim 4, characterized in that The step of obtaining the target prediction model based on the training of the sample data set includes: Grouping the sample data set to obtain multiple groups of sample data, wherein each group of sample data includes partial data sorted based on each data sorting rule; The target prediction model is obtained by training based on the multiple groups of sample data.

6. The method according to claim 5, characterized in that The target prediction model is obtained by training based on the multiple groups of sample data, including: For each group of sample data in the multiple groups of sample data, an initial prediction model is obtained by training the sample data, thereby obtaining multiple initial prediction models corresponding to the multiple groups of sample data; Scoring the performance of each initial prediction model in the multiple initial prediction models to obtain a scoring result for each initial prediction model; The target prediction model is determined according to the scoring results of the initial prediction models.

7. A data flow control system, characterized in that: include: Traffic prediction module and operation management module; The traffic prediction module is used to obtain target medical data traffic information within the current time period; Inputting the target medical data flow information into a target prediction model to obtain a target flow prediction result in a future time period, wherein the target prediction model is trained based on historical medical data flow information; The job management module is used to adjust the task processing resources used to process medical data based on the target flow prediction result and the flow threshold corresponding to the current time period.

8. A data flow control device, characterized in that: include: An acquisition unit, used to acquire target medical data flow information within a current time period; An input unit, used to input the target medical data flow information into a target prediction model to obtain a target flow prediction result in a future time period, wherein the target prediction model is trained based on historical medical data flow information; An adjustment unit is used to adjust the task processing resources used to process medical data based on the target flow prediction result and the flow threshold corresponding to the current time period.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 6 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images