A complex event asynchronous processing method and system based on cloud platform
By disassembling complex events into subtasks and dynamically scheduling computing resources using the elastic characteristics of the cloud platform, the traditional sequential processing of performance degradation in high concurrency situations is solved, and more efficient asynchronous processing and resource utilization are achieved, improving the cloud Platform responsiveness and throughput.
Patent Information
- Application Number
- CN202411252899.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-09-09
AI Technical Summary
Traditional sequential processing cannot respond quickly when facing high concurrency complex events, resulting in a decline in cloud platform performance and unable to effectively handle performance bottlenecks caused by large numbers of users and data.
Split complex events into multiple subtasks, create workflows, and dynamically schedule computing resources using the elastic characteristics of the cloud platform to ensure resource allocation of main programs and asynchronous transactions, perform corresponding operations according to the status code of the workflow node, and finally Send a notification of completion of processing.
Improves the system throughput and responsiveness, reduces latency, decouples system components, optimizes the utilization of computing resources, and avoids performance degradation.
Smart Images

Figure CN118779086B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a complex event asynchronous processing method and system based on a cloud platform. Background Art
[0002] As cloud platform services continue to expand, their users and data volumes will continue to increase, and some highly concurrent complex events will inevitably occur. Traditional sequential processing cannot respond quickly to these complex events, resulting in call blocking and a decline in the overall performance of the cloud platform. Asynchronous processing methods allow certain operations to be performed in the background without blocking the main program flow, which means that the program can continue to perform other tasks while waiting for long-running tasks to complete. This can avoid performance degradation on the cloud platform and improve the responsiveness of the application. Summary of the invention
[0003] The present invention provides a complex event asynchronous processing method based on a cloud platform, comprising:
[0004] Step 1: Break down complex events into multiple subtasks;
[0005] Step 2: Create a workflow for the complex event based on the decomposed subtasks;
[0006] Step 3: Use the elasticity of the cloud platform to dynamically schedule the computing resources of the workflow and main program;
[0007] Step 4: Execute the associated operation behavior according to the status code returned by each process node of the workflow;
[0008] Step 5. After all processes in the workflow are executed, a notification of the completion of complex event processing is sent to the staff.
[0009] The above-mentioned method for asynchronously processing complex events based on a cloud platform, wherein the complex event is decomposed into multiple subtasks, specifically includes the following sub-steps:
[0010] Identify the key steps in handling complex events;
[0011] Decompose complex events into several subtasks based on the identified key steps.
[0012] The above-mentioned method for asynchronously processing complex events based on a cloud platform, wherein a workflow is created for the complex event based on the decomposed subtasks, specifically including the following sub-steps:
[0013] Determine dependencies between subtasks in complex events;
[0014] Draw a workflow diagram based on the dependencies between subtasks;
[0015] Configure task properties for each process node in the workflow;
[0016] Integrate the created workflow on the cloud platform.
[0017] The above-mentioned complex event asynchronous processing method based on the cloud platform, in which the elastic characteristics of the cloud platform are used to dynamically schedule the computing resources of the workflow and the main program, specifically includes the following sub-steps:
[0018] Collect historical data on the main program's demand for computing resources on each service node of the cloud platform;
[0019] Create a main program demand prediction model based on the main program's historical demand data for computing resources;
[0020] Dynamic scheduling of computing resources is performed based on the prediction results of the main program demand prediction model.
[0021] The above-mentioned complex event asynchronous processing method based on the cloud platform, in which the dynamic scheduling of computing resources is performed based on the prediction result of the main program demand prediction model, specifically includes the following sub-steps:
[0022] Allocate initial computing resources for the main program and asynchronous transactions;
[0023] When the prediction result of the main program demand prediction model is lower than the initial computing resource allocation of the main program, the idle computing resources of the main program at the next time point are transferred to the asynchronous transaction;
[0024] When the prediction result of the main program demand prediction model is higher than the current computing resource ownership of the main program, the computing resources of the asynchronous transaction at the next time point are called to meet the main program demand first.
[0025] The above-mentioned complex event asynchronous processing method based on the cloud platform, in which the associated operation behavior is executed according to the status code returned by each process node of the workflow, specifically includes the following sub-steps:
[0026] Set different status codes for process nodes to indicate different execution states;
[0027] Create a process strategy table to associate different status codes with their corresponding operation behaviors;
[0028] Monitor the status code of each process node and execute the corresponding operation behavior.
[0029] The present invention also provides a complex event asynchronous processing system based on a cloud platform, comprising: a complex event splitting module, a workflow integration module, a computing resource dynamic scheduling module, a process control module and a notification module;
[0030] Complex event splitting module, used to split complex events into multiple subtasks;
[0031] Workflow integration module, which is used to create a workflow for complex events based on the decomposed subtasks and integrate it on the cloud platform;
[0032] The computing resource dynamic scheduling module is used to dynamically schedule the computing resources of the workflow and the main program;
[0033] The process control module is used to execute the associated operation behaviors according to the status of each process node of the workflow;
[0034] The notification module is used to send a notification of the completion of complex event processing to the staff after the workflow is executed.
[0035] A cloud platform-based complex event asynchronous processing system as described above, wherein the complex event splitting module includes a key step identification submodule and a subtask creation submodule;
[0036] The key step identification submodule is used to identify the key steps in processing complex events;
[0037] The subtask creation submodule is used to decompose complex events into several subtasks based on the identified key steps.
[0038] A complex event asynchronous processing system based on a cloud platform as described above, wherein the workflow integration module includes a flowchart drawing submodule and a process node configuration submodule;
[0039] A flowchart drawing submodule is used to draw a workflow diagram based on the dependency relationship of subtasks;
[0040] The process node configuration submodule is used to configure task properties for each process node in the workflow.
[0041] A complex event asynchronous processing system based on a cloud platform as described above, wherein the computing resource dynamic scheduling module includes a main program demand prediction submodule, a dynamic scheduling submodule and a computing resource monitoring submodule;
[0042] The main program demand prediction submodule is used to use the main program demand prediction model to predict the computing resource demand of the main program at the next time point;
[0043] The dynamic scheduling submodule is used to dynamically schedule computing resources based on the prediction results of the main program requirements;
[0044] The computing resource monitoring submodule is used to monitor the computing resource usage of the main program.
[0045] The beneficial effects achieved by the present invention are as follows: using an asynchronous method to process complex events can improve system throughput, reduce latency, and decouple system components to better utilize system computing resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0047] Figure 1 This is a flow chart of a complex event asynchronous processing method based on a cloud platform provided in Example 1 of the present invention. DETAILED DESCRIPTION
[0048] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0049] Embodiment 1
[0050] like Figure 1 As shown, the first embodiment of the present invention provides a complex event asynchronous processing method based on a cloud platform, including:
[0051] Step S10: Decompose the complex event into multiple subtasks;
[0052] To ensure that the execution of the main program of the cloud platform is not affected, sufficient resources should be reserved for the main program when processing complex events, and the remaining resources should be used to execute complex events. Under the condition of limited resource utilization, complex events need to be decomposed into multiple subtasks to avoid the execution body being too large and blocking the process. Specifically:
[0053] Step S11: Identify key steps for processing complex events;
[0054] The key steps are identified by the data flow trends and data life cycle in the process of complex event history processing. Each key step has a clear data flow in and out, and ensures that each key step only does one thing.
[0055] Step S12: Decompose the complex event into several subtasks according to the identified key steps;
[0056] The specific implementation method of each key step is a subtask, and its data flow is the input and output data of the subtask. These disassembled subtasks are stored in a message queue, which is used to transmit the subtasks.
[0057] Step S20: creating a workflow for the complex event based on the decomposed subtasks;
[0058] The starting point and end point of this workflow are the triggering condition and expected result of the complex event respectively. When the workflow is executed, the complex event is also processed. The creation process of the workflow specifically includes the following sub-steps:
[0059] Step S21: Determine the dependency relationship of subtasks in the complex event;
[0060] The input and output of subtasks can be used to determine whether there is a dependency between subtasks. If the input of subtask A is the output of subtask B, it means that A depends on B, and A can only be executed after B is executed.
[0061] Step S22: Draw a workflow diagram based on the dependency relationship of the subtasks;
[0062] In the figure, each subtask is represented as a process node, and the arrows between the nodes indicate their flow relationship. Subtasks that can be executed independently are processed in parallel, while subtasks with dependencies need to be marked with arrows to indicate their execution order.
[0063] Step S23: configuring task attributes for each process node in the workflow;
[0064] Task attributes include: execution conditions, execution expectations, execution frequency, timeout, and number of retries. These attributes are used to control the execution of subtasks on process nodes. The timeout refers to the expected execution time of the subtask. The expected execution time of the subtask is estimated according to the following formula:
[0065] ,in is the expected execution time of the kth subtask, Indicates the remaining amount of current computing resources except the main program. Indicates the total amount of computing resources, represents the competition coefficient of the kth subtask, Indicates the weight of the impact of computing resources on the execution time of tasks, The amount of data processed by the kth subtask, is the mean of the historical data volume processed by the k-th subtask, Indicates the weight of the impact of the amount of data processed on the time consumed in task execution. Indicates the current rate of network I / O operations. represents the average network I / O operation rate of the kth subtask in the historical processing record, Indicates the weight of the impact of network I / O operation rate on task execution time. Represents the average execution time of the kth subtask in the historical processing records.
[0066] Step S24: Integrate the created workflow on the cloud platform;
[0067] After the workflow of the complex event is integrated into the cloud platform, the complex event will be executed asynchronously according to the process set in the workflow when it is encountered later, without blocking the normal operation of the main program.
[0068] Step S30: dynamically schedule the computing resources of the workflow and the main program using the elasticity of the cloud platform;
[0069] The asynchronous execution of workflows can ensure the normal operation of the main program. When the main program has less demand for computing resources, the cloud platform should be able to dispatch idle resources to the workflow, thereby improving the processing efficiency of complex events. Specifically:
[0070] Step S31: Count the historical demand data of the main program for computing resources of each service node of the cloud platform;
[0071] The historical demand for computing resources of the main program is serialized by time to obtain the computing resource demand of the main program on each service node at different time points. Then, the computing resource demand of each service node is summed up to obtain the computing resource demand of the main program at different time points. After being organized into a data set, it is recorded as ,in ~ They represent the total computing resource requirements of the main program at different timestamps, and z is the total number of timestamps.
[0072] Step S32: creating a main program demand prediction model based on the historical demand data of the main program for computing resources;
[0073] The main program demand forecasting model is expressed as:
[0074] ,in Calculate the predicted resource requirements for the main program at the next time point. represents the smoothing parameter of the model, represents the total demand for computing resources by the main program at the tth timestamp in the input set, Indicates the trend parameter of the t timestamp, Indicates the total demand for computing resources by the main program at the starting timestamp of the previous change cycle. is the trend parameter of the current change cycle, Indicates the first The total demand for computing resources by the main program at each timestamp is: It represents the trend parameter of the previous change cycle, h is the length of the previous change cycle, and m is the length of the current change cycle. represents the total demand at the t-1 timestamp in the input set, Indicates the trend parameter of the t-1 timestamp.
[0075] The dataset C is used as training data to train the main program demand prediction model. During the training process, various parameters of the model are adjusted to make the prediction accuracy reach more than 90%.
[0076] Step S33: dynamically scheduling computing resources based on the prediction results of the main program demand prediction model;
[0077] Set a timer to automatically collect the computing resource requirements of the main program at different timestamps in the last N hours, organize them into input sets and input them into the main program demand prediction model, and output the computing resource requirements of the main program at the next time point. The timer trigger time, trigger conditions, and collection and output cycle are all set according to specific needs. Based on this prediction result, the computing resources on the cloud platform service node are scheduled in real time to improve the execution efficiency of asynchronous transactions. The specific scheduling method includes the following sub-steps:
[0078] Step S331: Allocate initial computing resources to the main program and the asynchronous transaction;
[0079] The workflow of complex events is executed asynchronously, so it is directly called asynchronous transaction here. The initial computing resources allocated to the main program can refer to the average demand of the main program in the previous day, and the remaining amount is used as the initial computing resources of the asynchronous transaction.
[0080] Step S332: when the prediction result of the main program demand prediction model is lower than the initial computing resource allocation of the main program, the idle computing resources of the main program at the next time point are transferred to the asynchronous transaction;
[0081] The idle computing resources of the main program at the next time point are obtained by subtracting the predicted resources from the initial computing resources of the main program.
[0082] Step S333: when the prediction result of the main program demand prediction model is higher than the current computing resource ownership of the main program, the computing resources of the asynchronous transaction at the next time point are called to preferentially meet the main program demand;
[0083] The amount of asynchronous transaction computing resources to be called at the next time point L is calculated according to the following formula:
[0084] ,in Indicates the predicted amount of computing resources for the main program, Indicates the initial amount of computing resources for the main program. Indicates the current amount of asynchronous transaction computing resources. Indicates the initial amount of asynchronous transaction computing resources. Return ,when return .
[0085] Step S40: executing the associated operation behavior according to the status code returned by each process node of the workflow;
[0086] During the execution of a process node, various status codes are returned. Based on these status codes, it can be determined whether the process node has been completed or what problems have been encountered, so as to perform the next operation. Specifically:
[0087] Step S41: setting different status codes for process nodes to indicate different execution states;
[0088] The status code is in the form of a prompt word plus a number. The prompt word is used to help staff quickly locate the problem, and the number allows the machine to quickly identify it. For example, when the execution time of the kth process node exceeds the expected execution time When the number of requests reaches 30%, a timeout status code is returned. The timeout status code can be represented by "Timeout-001".
[0089] Step S42: Establish a process strategy table to associate different status codes with their corresponding operation behaviors;
[0090] The process strategy table maintains the operation behaviors associated with each status code, including retrying, pausing execution, opening or closing certain services, changing backup ports, etc. to deal with different execution states.
[0091] Step S43: monitor the status code of each process node and execute its corresponding operation behavior.
[0092] Step S50: After all processes in the workflow are executed, a complex event processing completion notification is sent to the staff;
[0093] When the status codes of all process nodes in the workflow are completed, the complex event is processed and then notified to the staff through a system message, and the staff will proceed to the next step.
[0094] Embodiment 2
[0095] Embodiment 2 of the present invention provides a complex event asynchronous processing system based on a cloud platform, comprising: a complex event splitting module, a workflow integration module, a computing resource dynamic scheduling module, a process control module and a notification module;
[0096] (1) Complex event splitting module, which is used to split complex events into multiple subtasks, including a key step identification submodule and a subtask creation submodule;
[0097] 1. Key step identification submodule, used to identify the key steps in processing complex events;
[0098] The key steps are identified by the data flow trends and data life cycle in the process of complex event history processing. Each key step has a clear data flow in and out, and ensures that each key step only does one thing.
[0099] 2. Subtask creation submodule, which is used to decompose complex events into several subtasks based on the identified key steps;
[0100] The specific implementation method of each key step is a subtask, and its data flow is the input and output data of the subtask. These disassembled subtasks are stored in a message queue, which is used to transmit the subtasks.
[0101] (2) Workflow integration module, which is used to create a workflow for complex events based on the decomposed subtasks and integrate it on the cloud platform, including a flowchart drawing submodule and a process node configuration submodule;
[0102] 1. Flowchart drawing submodule, used to draw workflow diagrams based on the dependencies of subtasks;
[0103] The starting point and end point of the workflow are the triggering conditions and expected results of the complex event respectively. The input and output of the subtasks can be used to determine whether there is a dependency between the subtasks. If the input of subtask A is the output of subtask B, it means that A depends on B, and A can only be executed after B is executed.
[0104] In the figure, each subtask is represented as a process node, and the arrows between the nodes indicate their flow relationship. Subtasks that can be executed independently are processed in parallel, while subtasks with dependencies need to be marked with arrows to indicate their execution order.
[0105] 2. The process node configuration submodule is used to configure task attributes for each process node in the workflow;
[0106] Task attributes include: execution conditions, execution expectations, execution frequency, timeout, and number of retries. These attributes are used to control the execution of subtasks on process nodes. The timeout refers to the expected execution time of the subtask. The expected execution time of the subtask is estimated according to the following formula:
[0107] ,in is the expected execution time of the kth subtask, Indicates the remaining amount of current computing resources except the main program. Indicates the total amount of computing resources, represents the competition coefficient of the kth subtask, Indicates the weight of the impact of computing resources on the execution time of tasks, The amount of data processed by the kth subtask, is the mean of the historical data volume processed by the k-th subtask, Indicates the weight of the impact of the amount of data processed on the time consumed in task execution. Indicates the current rate of network I / O operations. represents the average network I / O operation rate of the kth subtask in the historical processing record, Indicates the weight of the impact of network I / O operation rate on task execution time. Represents the average execution time of the kth subtask in the historical processing records.
[0108] After the workflow of the complex event is integrated into the cloud platform, the complex event will be executed asynchronously according to the process set in the workflow when it is encountered later, without blocking the normal operation of the main program.
[0109] (3) Computing resource dynamic scheduling module, which is used to dynamically schedule the computing resources of the workflow and the main program, including the main program demand prediction submodule, the dynamic scheduling submodule and the computing resource monitoring submodule;
[0110] 1. The main program demand prediction submodule is used to use the main program demand prediction model to predict the computing resource demand of the main program at the next time point;
[0111] The creation process of the main program demand forecasting model includes:
[0112] I. Count the historical demand of the main program for computing resources of each service node of the cloud platform;
[0113] The historical demand for computing resources of the main program is serialized by time to obtain the computing resource demand of the main program on each service node at different time points. Then, the computing resource demand of each service node is summed up to obtain the computing resource demand of the main program at different time points. After being organized into a data set, it is recorded as ,in ~ They represent the total computing resource requirements of the main program at different timestamps, and z is the total number of timestamps.
[0114] II. Train and optimize the main program demand prediction model based on the main program's historical demand data for computing resources;
[0115] The main program demand forecasting model is expressed as:
[0116] ,in Calculate the predicted resource requirements for the main program at the next time point. represents the smoothing parameter of the model, represents the total demand for computing resources by the main program at the tth timestamp in the input set, Indicates the trend parameter of the t timestamp, Indicates the total demand for computing resources by the main program at the starting timestamp of the previous change cycle. is the trend parameter of the current change cycle, Indicates the first The total demand for computing resources by the main program at each timestamp is: It represents the trend parameter of the previous change cycle, h is the length of the previous change cycle, and m is the length of the current change cycle. represents the total demand at the t-1 timestamp in the input set, Indicates the trend parameter of the t-1 timestamp.
[0117] The dataset C is used as training data to train the main program demand prediction model. During the training process, various parameters of the model are adjusted to make the prediction accuracy reach more than 90%.
[0118] 2. Dynamic scheduling submodule, used to dynamically schedule computing resources based on the prediction results of the main program requirements;
[0119] Set a timer to automatically collect the computing resource requirements of the main program at different timestamps in the last N hours, organize them into input sets and input them into the main program demand prediction model, and output the computing resource requirements of the main program at the next time point. The timer trigger time, trigger conditions, and collection and output cycle are all set according to specific needs. Based on this prediction result, the computing resources on the cloud platform service node are scheduled in real time to improve the execution efficiency of asynchronous transactions. The specific scheduling methods include:
[0120] I. Allocate initial computing resources for the main program and asynchronous transactions;
[0121] The workflow of complex events is executed asynchronously, so it is directly called asynchronous transaction here. The initial computing resources allocated to the main program can refer to the average demand of the main program in the previous day, and the remaining amount is used as the initial computing resources of the asynchronous transaction.
[0122] II. When the prediction result of the main program demand prediction model is lower than the initial computing resource allocation of the main program, the idle computing resources of the main program at the next time point are transferred to the asynchronous transaction;
[0123] The idle computing resources of the main program at the next time point are obtained by subtracting the predicted resources from the initial computing resources of the main program.
[0124] III. When the prediction result of the main program demand prediction model is higher than the current computing resource ownership of the main program, the computing resources of the asynchronous transaction at the next time point are called to meet the main program demand first;
[0125] The amount of asynchronous transaction computing resources to be called at the next time point L is calculated according to the following formula:
[0126] ,in Indicates the predicted amount of computing resources for the main program, Indicates the initial amount of computing resources for the main program. Indicates the current amount of asynchronous transaction computing resources. Indicates the initial amount of asynchronous transaction computing resources. Return ,when return .
[0127] 3. Computing resource monitoring submodule, used to monitor the computing resource usage of the main program.
[0128] (4) Process control module, which is used to execute the associated operation behaviors according to the status of each process node of the workflow, including process monitoring submodule, process strategy definition submodule and process strategy execution submodule;
[0129] 1. The process monitoring submodule is used to return the status code of each process node in the workflow of complex events;
[0130] The status code is in the form of a prompt word plus a number. The prompt word is used to help staff quickly locate the problem, and the number allows the machine to quickly identify it. For example, when the execution time of the kth process node exceeds the expected execution time When the number of requests reaches 30%, a timeout status code is returned. The timeout status code can be represented by "Timeout-001".
[0131] 2. The process strategy definition submodule is used to define the associated operation behaviors for different status codes;
[0132] The status code is associated with the operation behavior, including retrying, pausing execution, turning on or off certain services, changing backup ports, etc. It is used to respond to different execution states and tell the system what to do next.
[0133] 3. The process strategy execution submodule is used to execute the associated operation behavior on the process node according to the monitored process node status code.
[0134] (5) Notification module, used to send a notification of the completion of complex event processing to the staff after the workflow is executed;
[0135] When the status codes of all process nodes in the workflow are completed, the complex event is processed and then notified to the staff via system messages, who then proceed to the next step.
[0136] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.
Claims
1. A complex event asynchronous processing method based on a cloud platform, characterized in that: The method comprises: Step 1: Break down complex events into multiple subtasks; Step 2: Create a workflow for the complex event based on the decomposed subtasks; Each process node in the workflow is configured with detailed task properties to control the execution process of the subtasks on the process node, including execution conditions, execution expectations, execution frequency, timeout, and number of retries. The timeout refers to the expected execution time of the subtask. The expected execution time of the subtask is estimated according to the following formula: ,in is the expected execution time of the kth subtask, Indicates the remaining amount of current computing resources except the main program. Indicates the total amount of computing resources, represents the competition coefficient of the kth subtask, Indicates the weight of the impact of computing resources on the execution time of tasks, The amount of data processed by the kth subtask, is the mean of the historical data volume processed by the k-th subtask, Indicates the weight of the impact of the amount of data processed on the time consumed in task execution. Indicates the current rate of network I / O operations. represents the average network I / O operation rate of the kth subtask in the historical processing record, Indicates the weight of the impact of network I / O operation rate on task execution time. It represents the average execution time of the k-th subtask in the historical processing records; Step 3: Use the elasticity of the cloud platform to dynamically schedule the computing resources of the workflow and main program. This is divided into the following sub-steps: Collect historical data on the main program's demand for computing resources on each service node of the cloud platform; Create a main program demand prediction model based on the main program's historical demand data for computing resources; The main program demand forecasting model is expressed as: ,in Calculate the predicted resource requirements for the main program at the next time point. represents the smoothing parameter of the model, represents the total demand for computing resources by the main program at the tth timestamp in the input set, Indicates the trend parameter of the t timestamp, Indicates the total demand for computing resources by the main program at the starting timestamp of the previous change cycle. is the trend parameter of the current change cycle, Indicates the first The total demand for computing resources by the main program at each timestamp is: It represents the trend parameter of the previous change cycle, h is the length of the previous change cycle, and m is the length of the current change cycle. represents the total demand at the t-1 timestamp in the input set, Indicates the trend parameter of t-1 timestamp; Dynamically schedule computing resources based on the prediction results of the main program demand prediction model; Step 4: Execute the associated operation behavior according to the status code returned by each process node of the workflow, which is divided into the following sub-steps: Set different status codes for process nodes to indicate different execution states; Create a process strategy table to associate different status codes with their corresponding operation behaviors; Monitor the status code of each process node and execute the corresponding operation behavior; Step 5. After all processes in the workflow are executed, a notification of the completion of complex event processing is sent to the staff.
2. The method for asynchronously processing complex events based on a cloud platform according to claim 1, characterized in that: Decompose complex events into multiple subtasks, including the following sub-steps: Identify key steps in handling complex events; Decompose complex events into several subtasks based on the identified key steps.
3. The method for asynchronously processing complex events based on a cloud platform according to claim 1, characterized in that: Create a workflow for a complex event based on the decomposed subtasks, which includes the following sub-steps: Determine dependencies between subtasks in complex events; Draw a workflow diagram based on the dependencies between subtasks; Configure task properties for each process node in the workflow; Integrate the created workflow on the cloud platform.
4. The method for asynchronously processing complex events based on a cloud platform according to claim 1, characterized in that: Dynamic scheduling of computing resources based on the prediction results of the main program demand prediction model includes the following sub-steps: Allocate initial computing resources for the main program and asynchronous transactions; When the prediction result of the main program demand prediction model is lower than the initial computing resource allocation of the main program, the idle computing resources of the main program at the next time point are transferred to the asynchronous transaction; When the prediction result of the main program demand prediction model is higher than the current computing resource ownership of the main program, the computing resources of the asynchronous transaction at the next time point are called to meet the main program demand first.
5. A complex event asynchronous processing system based on a cloud platform, characterized in that: The system includes: a complex event splitting module, a workflow integration module, a computing resource dynamic scheduling module, a process control module and a notification module; Complex event splitting module, used to split complex events into multiple subtasks; Workflow integration module, which is used to create a workflow for complex events based on the decomposed subtasks and integrate it on the cloud platform; Each process node in the workflow is configured with detailed task properties to control the execution process of the subtasks on the process node, including execution conditions, execution expectations, execution frequency, timeout, and number of retries. The timeout refers to the expected execution time of the subtask. The expected execution time of the subtask is estimated according to the following formula: ,in is the expected execution time of the kth subtask, Indicates the remaining amount of current computing resources except the main program. Indicates the total amount of computing resources, represents the competition coefficient of the kth subtask, Indicates the weight of the impact of computing resources on the execution time of tasks, The amount of data processed by the kth subtask, is the mean of the historical data volume processed by the k-th subtask, Indicates the weight of the impact of the amount of data processed on the time consumed in task execution. Indicates the current rate of network I / O operations. represents the average network I / O operation rate of the kth subtask in the historical processing record, Indicates the weight of the impact of network I / O operation rate on task execution time. It represents the average execution time of the k-th subtask in the historical processing records; The computing resource dynamic scheduling module is used to dynamically schedule the computing resources of the workflow and the main program, including: Collect historical data on the main program's demand for computing resources on each service node of the cloud platform; Create a main program demand prediction model based on the main program's historical demand data for computing resources; The main program demand forecasting model is expressed as: ,in Calculate the predicted resource requirements for the main program at the next time point. represents the smoothing parameter of the model, represents the total demand for computing resources by the main program at the tth timestamp in the input set, Indicates the trend parameter of the t timestamp, Indicates the total demand for computing resources by the main program at the starting timestamp of the previous change cycle. is the trend parameter of the current change cycle, Indicates the first The total demand for computing resources by the main program at each timestamp is: It represents the trend parameter of the previous change cycle, h is the length of the previous change cycle, and m is the length of the current change cycle. represents the total demand at the t-1 timestamp in the input set, Indicates the trend parameter of t-1 timestamp; Dynamically schedule computing resources based on the prediction results of the main program demand prediction model; The process control module is used to execute the associated operation behaviors according to the status of each process node of the workflow, including: Set different status codes for process nodes to indicate different execution states; Create a process strategy table to associate different status codes with their corresponding operation behaviors; Monitor the status code of each process node and execute the corresponding operation behavior; The notification module is used to send a notification of the completion of complex event processing to the staff after the workflow is executed.
6. A cloud platform-based complex event asynchronous processing system according to claim 5, characterized in that: Complex event splitting module, including key step identification submodule and subtask creation submodule; The key step identification submodule is used to identify the key steps in processing complex events; The subtask creation submodule is used to decompose complex events into several subtasks based on the identified key steps.
7. A cloud platform-based complex event asynchronous processing system according to claim 5, characterized in that: The workflow integration module includes a flowchart drawing submodule and a process node configuration submodule; A flowchart drawing submodule is used to draw a workflow diagram based on the dependency relationship of subtasks; The process node configuration submodule is used to configure task properties for each process node in the workflow.
8. The cloud platform-based complex event asynchronous processing system according to claim 5, characterized in that: The computing resource dynamic scheduling module includes a main program demand prediction submodule, a dynamic scheduling submodule and a computing resource monitoring submodule; The main program demand prediction submodule is used to use the main program demand prediction model to predict the computing resource demand of the main program at the next time point; The dynamic scheduling submodule is used to dynamically schedule computing resources based on the prediction results of the main program requirements; The computing resource monitoring submodule is used to monitor the computing resource usage of the main program.
Citation Information
Patent Citations
Video analysis and accelerating method based on thread level flow line
CN106358003A
Multi-user multi-priority distributed cooperative processing method based on cloud computing
CN111367630A
CPU-GPU (Central Processing Unit-Graphic Processing Unit) cooperative scheduling method and device based on asynchronous operation
CN117032999A
Scheduling method for high-concurrency processing of small tasks in distributed cluster
CN118394476A
