Task processing method and device based on state machine

Abstract

The application provides a state machine-based task processing method and device, wherein the state machine-based task processing method comprises the following steps: obtaining a target state instance of a target task, and determining a target state machine according to the target state instance, wherein the target state machine comprises a state instance queue and an instance reprocessing queue; adding the target state instance to the state instance queue; pulling the target state instance from the state instance queue, performing instance processing on the target state instance based on a state machine mode of the target state machine, and obtaining a processing result; in the case that the processing result is state jump unsuccessful, adding the target state instance to the instance reprocessing queue; adding the target state instance in the instance reprocessing queue to the state instance queue, and continuing to perform the operation of pulling the target state instance from the state instance queue, and performing instance processing on the target state instance based on the state machine mode of the target state machine.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a task processing method based on a state machine. This application also relates to a task processing apparatus based on a state machine, a computing device, and a computer-readable storage medium. Background Technology

[0002] Task scheduling is a common mechanism for program implementation. For tasks with a lot of processes, the task is usually broken down into state machines. Each state machine corresponds to a step. State machines have mutual dependencies and may jump, rotate, or perform other behaviors. Each state machine of a task may also fail to execute at a certain time.

[0003] Analyzing the tasks themselves, some tasks require asynchronous calls, meaning that after adding the task to the scheduler, you only need to wait for it to complete. Other tasks are more time-sensitive and require setting a time period at the beginning. If this period is exceeded, a timeout message needs to be returned for the task.

[0004] Current state machine-based task scheduling methods require business personnel to retry manually after execution failures, which is cumbersome and has limited functionality. Summary of the Invention

[0005] In view of this, embodiments of this application provide a task processing method based on a state machine. This application also relates to a task processing apparatus based on a state machine, a computing device, and a computer-readable storage medium, to solve the problem that in existing state machine-based task scheduling methods, business personnel need to manually retry after execution failure.

[0006] According to a first aspect of the embodiments of this application, a task processing method based on a state machine is provided, including:

[0007] Obtain a target state instance of the target task, and determine a target state machine based on the target state instance, wherein the target state machine includes a state instance queue and an instance reprocessing queue;

[0008] Add the target state instance to the state instance queue;

[0009] The target state instance is retrieved from the state instance queue, and the target state instance is processed based on the state machine pattern of the target state machine to obtain the processing result.

[0010] If the processing result is that the state transition is unsuccessful, the target state instance is added to the instance reprocessing queue;

[0011] The target state instance in the instance reprocessing queue is added to the state instance queue, and the operation of pulling the target state instance from the state instance queue and performing instance processing on the target state instance based on the state machine pattern of the target state machine continues.

[0012] According to a second aspect of the embodiments of this application, a task processing apparatus based on a state machine is provided, comprising:

[0013] The acquisition module is configured to acquire a target state instance of a target task and determine a target state machine based on the target state instance, wherein the target state machine includes a state instance queue and an instance reprocessing queue.

[0014] The instance queue addition module is configured to add the target state instance to the state instance queue.

[0015] The processing module is configured to pull the target state instance from the state instance queue, perform instance processing on the target state instance based on the state machine pattern of the target state machine, and obtain the processing result;

[0016] The reprocessing queue addition module is configured to add the target state instance to the instance reprocessing queue if the processing result is that the state transition is unsuccessful.

[0017] The reprocessing module is configured to add the target state instance from the instance reprocessing queue to the state instance queue, and continue to perform the operation of pulling the target state instance from the state instance queue and performing instance processing on the target state instance based on the state machine pattern of the target state machine.

[0018] According to a third aspect of the embodiments of this application, a computing device is provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor executes the computer instructions to implement the steps of the state machine-based task processing method.

[0019] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided that stores computer instructions, which, when executed by a processor, implement the steps of the state machine-based task processing method.

[0020] The state machine-based task processing method provided in this application obtains a target state instance of a target task and determines a target state machine based on the target state instance. The target state machine includes a state instance queue and an instance reprocessing queue. The method involves adding the target state instance to the state instance queue, retrieving the target state instance from the state instance queue, performing instance processing on the target state instance based on the state machine pattern of the target state machine, and obtaining a processing result. If the processing result indicates that the state transition was unsuccessful, the method adds the target state instance to the instance reprocessing queue. The method then adds the target state instance from the instance reprocessing queue back to the state instance queue and continues to perform the operation of retrieving the target state instance from the state instance queue and performing instance processing on the target state instance based on the state machine pattern of the target state machine.

[0021] One embodiment of this application implements the processing of target state instances of a target task through a state instance queue and an instance reprocessing queue. The target state instance is added to the state instance queue and executed through the state instance queue. If the processing result of the target state instance is that the state transition is unsuccessful, the target state instance is added to the instance reprocessing queue and waits to be called for processing again, thus realizing the functions of task failure retry and self-loop. Attached Figure Description

[0022] Figure 1 This is a flowchart of a task processing method based on a state machine provided in an embodiment of this application;

[0023] Figure 2 This is a framework diagram of an intelligent scheduling framework provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram illustrating the flow of a synchronous state machine according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the flow of an asynchronous state machine provided in an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of a migration task process provided in an embodiment of this application;

[0027] Figure 6 This is a schematic diagram of the structure of a state machine-based task processing device provided in an embodiment of this application;

[0028] Figure 7 This is a structural block diagram of a computing device provided in one embodiment of this application. Detailed Implementation

[0029] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0030] The terminology used in one or more embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this application. The singular forms “a,” “the,” and “the” used in one or more embodiments of this application and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” used in one or more embodiments of this application refers to and includes any or all possible combinations of one or more associated listed items.

[0031] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this application, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0032] First, the terms and concepts involved in one or more embodiments of this application will be explained.

[0033] Task status: A task can be broken down into multiple steps, and each step is a task status.

[0034] Task status execution type: Each task can be divided into synchronous or asynchronous. Synchronous tasks are those that directly enter the main thread and are queued for execution, while asynchronous tasks are those that do not enter the main thread but enter the task queue to wait to be called.

[0035] State machine: A state machine consists of a state register and combinational logic circuits. It can transition between states according to a pre-set state based on control signals. It is the control center that coordinates related signal actions and completes specific operations.

[0036] In the design of complex systems (such as distributed object storage systems, microservices, etc.), the processes of certain tasks are very complex. The execution of a task may be divided into multiple steps, and each step handles different events, including but not limited to sending RPC requests, performing database operations, processing business computing logic in real time, asynchronously waiting for other component calls, and processing the same event multiple times in a loop based on conditions. The execution of these tasks needs to face some relatively common execution logic, such as state machine transitions, retries, and sequential parallel execution after queuing. If these logics are coupled in the task implementation process, it will lead to a very complex process and poor reusability.

[0037] Analyzing the tasks themselves, some tasks require asynchronous calls, meaning that after adding the task to the scheduler, you only need to wait for it to complete. Other tasks are more time-sensitive and require setting a time period at the beginning. If this period is exceeded, a timeout message needs to be returned for the task.

[0038] Task scheduling is a common mechanism for program implementation. For tasks with a lot of processes, the task is usually broken down into state machines. Each state machine corresponds to a step. State machines have mutual dependencies and may jump, rotate, or perform other behaviors. Each state machine of a task may also fail to execute at a certain time.

[0039] Current state machine-based task scheduling methods have the following problems:

[0040] 1. After a state machine fails to execute, the business personnel or the caller need to retry it themselves.

[0041] 2. The state machine's self-loop requires execution via business logic calls.

[0042] 3. Unable to perform rate limiting control on certain types of tasks and state machines.

[0043] 4. The state machine cannot support asynchronous mode, or the asynchronous mode state machine cannot perform secure authentication external calls.

[0044] Based on this, this application provides a task processing method based on a state machine. This application also relates to a task processing device based on a state machine, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0045] Figure 1 A flowchart of a state machine-based task processing method according to an embodiment of this application is shown, which specifically includes the following steps:

[0046] Step 102: Obtain the target state instance of the target task, and determine the target state machine based on the target state instance, wherein the target state machine includes a state instance queue and an instance reprocessing queue.

[0047] The target task specifically refers to any one of several task categories. The task category specifically refers to a semantically meaningful event in the real world, such as ordering takeout or traveling around the world.

[0048] The state machine-based task processing method provided in this application relies on an intelligent scheduling framework, see [link to relevant documentation]. Figure 2 , Figure 2 A framework diagram of an intelligent scheduling framework provided in an embodiment of this application is shown, as follows: Figure 2 As shown, this intelligent scheduling framework includes an intelligent scheduler, a category scheduler, a database, and a state machine visualization interface. The intelligent scheduler (JobScheduler) supports running multiple categories of tasks simultaneously and applies rate limiting to each category. The category scheduler runs one or more tasks of the same category. The task category scheduler has different state transition modes for tasks, such as sequential calls, jumps, and self-loops, with each state corresponding to a state machine. The intelligent scheduler uses a persistence layer protocol to persistently store data with a single-machine or distributed database. Furthermore, the intelligent scheduler or database can read the transition data of any state machine and visualize it through the state machine visualization interface, allowing business personnel to understand the task status of target tasks in a timely manner.

[0049] A target state instance specifically refers to the running instance (job) of a target task in a target state. The target state specifically refers to the current subtask state corresponding to the current subtask. A target task may have multiple subtasks, and each subtask has its own subtask state. When each subtask is running, the execution instance corresponding to the current subtask state is the target state instance.

[0050] After obtaining the target state instance, it is necessary to determine the target state machine corresponding to the target state instance. In practical applications, each state machine includes a state instance queue and an instance reprocessing queue. The state instance queue is used to store state instances that jump from the previous task state to the current task state, and the instance reprocessing queue is used to store state instances in the target state machine that need to be executed again.

[0051] In practical applications, different target task states correspond to different target state machines. Specifically, determining the target state machine based on the target state instance includes:

[0052] Determine the task state execution type corresponding to the target state instance;

[0053] The state machine mode is determined based on the task state execution type, wherein the state machine mode includes synchronous mode and asynchronous mode;

[0054] The target state machine corresponding to the target state instance is determined based on the state machine pattern, wherein the state machine includes synchronous state machines and asynchronous state machines.

[0055] After obtaining the target state instance, it is necessary to obtain the task state execution type corresponding to the target state instance. The task state execution type is usually divided into synchronous and asynchronous. Therefore, based on the task state execution type, the state machine mode can be determined to be synchronous or asynchronous. When the state machine mode is synchronous, the target state machine of the target state instance should be determined to be a synchronous state machine; when the state machine mode is asynchronous, the target state machine of the target state instance should be determined to be an asynchronous state machine.

[0056] In one specific embodiment provided in this application, the target state instance job1 of the target task A is obtained. The task state execution type of the target state instance job1 is synchronous. Based on the synchronous type, the target state machine corresponding to the target state instance job1 is determined to be a synchronous state machine.

[0057] Correspondingly, if the task execution type of the target state instance job1 is asynchronous, then the target state machine corresponding to the target state instance job1 is an asynchronous state machine.

[0058] In practical applications, both synchronous and asynchronous state machines include a state instance queue and an instance reprocessing queue. Furthermore, the instance reprocessing queue includes a penalty queue and a loopback queue. The penalty queue stores state instances where calls failed, while the loopback queue stores state instances where calls succeeded but the transition conditions were not met.

[0059] Step 104: Add the target state instance to the state instance queue.

[0060] After obtaining the target state instance (job) and the target state machine corresponding to the target state instance, the target state instance can be added to the state instance queue in the target state machine.

[0061] In one specific embodiment provided in this application, taking job1 as the target state instance and synchronous state machine as the target state machine, job1 is added to the state instance queue (job queue) of the synchronous state machine.

[0062] In another specific embodiment provided in this application, taking job1 as the target state instance and asynchronous state machine as the target state machine, job1 is added to the state instance queue (job queue) of the asynchronous state machine.

[0063] Step 106: Retrieve the target state instance from the state instance queue, perform instance processing on the target state instance based on the state machine pattern of the target state machine, and obtain the processing result.

[0064] In the target state machine, state instances (jobs) are read sequentially from the state instance queue (job queue) for processing, and the jobs are processed according to the target state machine's state machine mode. The state instance queue follows a first-in, first-out (FIFO) reading rule; jobs entering the job queue are read out sequentially. In practical applications, the state machine mode includes synchronous or asynchronous modes. If the target state machine's state machine mode is synchronous, the job is processed according to the synchronous processing mode; if the target state machine's state machine mode is asynchronous, the job is processed according to the asynchronous processing mode.

[0065] The following sections will further explain how synchronous and asynchronous state machines process target state instances.

[0066] When the state machine mode of the target state machine is synchronous mode;

[0067] The target state instance is retrieved from the state instance queue, and instance processing is performed on the target state instance based on the state machine pattern of the target state machine to obtain the processing result, including:

[0068] The target state instance is invoked and processed through a task thread pool to obtain the invocation and processing information;

[0069] Obtain the jump condition corresponding to the target state instance;

[0070] If the call processing information does not meet the jump condition, the processing result is determined to be that the state jump was unsuccessful;

[0071] If the call processing information meets the jump conditions, the processing result is determined to be a successful state jump.

[0072] See Figure 3 , Figure 3 A schematic diagram illustrating the flow of an example of a synchronous state machine provided in an embodiment of this application is shown. For example... Figure 3As shown, the synchronous state machine includes a state instance queue (job queue) and an instance reprocessing queue (delay queue and self-circulating queue). The target state instance is placed into the job queue of the synchronous state machine, which is used to store state instances.

[0073] Synchronous state machines typically have a configurable task thread pool. This thread pool concurrently pulls jobs from the job queue for processing. Figure 3 In the task process pool, processes such as gorutine1 (process 1), gorutine2, etc., will sequentially pull jobs from the job queue for corresponding call processing and obtain call processing information. The call processing information is used to indicate the relevant processing result information of the job, such as whether the processing was successful. If the processing is successful, there will be relevant information about the success; if the processing fails, there will be relevant information about the failure.

[0074] The jump condition for a target state instance specifically refers to the conditions that must be met for the target state instance to jump to the next task state. For example, if the target state instance requires "successful execution 5 times in the current state", it means that the target state instance needs to be executed successfully 5 times before it can jump to the next task state. Therefore, the jump condition for the target state instance is "successful execution 5 times".

[0075] The job's call processing information will count its actual call processing content. Continuing with the previous example, if the job's call processing information is "call failed", it means that the call processing information meets the jump condition; if the job's call processing information is "successfully executed 3 times", it does not meet the jump condition "successfully executed 5 times", it means that the call processing information does not meet the jump condition; if the job's call processing information is "successfully executed 5 times", it meets the jump condition "successfully executed 5 times", and the call processing information meets the jump condition.

[0076] The following explanation uses an asynchronous state machine as an example, assuming the target state machine's state machine mode is asynchronous.

[0077] The target state instance is retrieved from the state instance queue, and instance processing is performed on the target state instance based on the state machine pattern of the target state machine to obtain the processing result, including:

[0078] Add the target state instance to the instance cache corresponding to the target state machine;

[0079] Generate call information corresponding to the target state instance in the notification queue, wherein the call information includes the location information of the target state instance in the instance cache;

[0080] Receive the processing result returned by the asynchronous call based on the call information.

[0081] See Figure 4 , Figure 4 A schematic diagram illustrating the flow of an example of an asynchronous state machine provided in an embodiment of this application is shown. For example... Figure 4 As shown, asynchronous state machines also include a state instance queue (job queue) and an instance reprocessing queue (delay queue and self-circulating queue). The state instance queue (job queue) and instance reprocessing queue (delay queue and self-circulating queue) in an asynchronous state machine have the same function and process as those in a synchronous state machine.

[0082] Unlike synchronous state machines, asynchronous state machines do not use thread pools to process jobs. Instead, they incorporate an instance cache (job cache) and a notification queue. Asynchronous job calls are implemented through the job cache and notification queue. The job cache is a non-evident job cache. When a job enters an asynchronous state, it is read from the job queue and written to the job cache. Simultaneously, a call message that can be asynchronously invoked is generated and written to the notification queue. This call message includes the job's job identifier (jobID) and its position in the job cache.

[0083] The external environment of the state machine can learn which jobs can be called through the notification queue, and the external environment can realize the asynchronous call to the job through the call information in the notification queue, and after the call is completed, it will feed back the processing result to the asynchronous state machine.

[0084] Specifically, receiving the processing result returned by the asynchronous call based on the call information includes:

[0085] Receive an asynchronous calling program's calling instruction in response to the calling information, wherein the calling instruction includes a task identifier and a task tag;

[0086] Based on the location information, the target state instance is retrieved asynchronously and the target state instance is marked with the task tag.

[0087] Receive the call processing information of the asynchronous call and obtain the jump condition corresponding to the target state instance;

[0088] If the call processing information does not meet the jump condition, the processing result is determined to be that the state jump was unsuccessful;

[0089] If the call processing information meets the jump conditions, the processing result is determined to be a successful state jump.

[0090] In practical applications, the external environment (asynchronous calling program) uses the TryGetJobState interface to send a notification queue retrieval instruction to the asynchronous state machine through the asynchronous calling module. This retrieval instruction can retrieve which job call information is ready in the notification queue. Based on the call information of the job to be called, a call instruction is issued. The call instruction includes the jobID (job identifier) ​​and tag (task label) of the job to be called. The job is retrieved from the job cache through the job location information recorded in the call information to realize the asynchronous call, and the job is tagged (task label) at the same time.

[0091] In practical applications, the jobID+tag calling method is used to enhance the security of asynchronous calls from external environments. The jobID is used to identify the job, and the tag is used for authentication. For example, if two different external environments call the same job at the same time, when the external environment completes its call and sends the result back to the asynchronous state machine, the same job will have two call results. The asynchronous state machine cannot know the correspondence between the call result and the external environment. Therefore, the tag ensures the consistency between the external environment obtaining the job and returning the call result, thereby preventing dirty data from being generated in the asynchronous state machine.

[0092] The external environment can retrieve the job from the job cache through asynchronous calls, and then perform asynchronous call processing. After processing is completed, the generated call processing information will be returned to the asynchronous state machine, where the job's jump conditions will also be obtained.

[0093] After obtaining the call processing information and jump conditions, the processing logic in the asynchronous state machine is the same as that in the synchronous state machine. If the call processing information meets the jump conditions, the job's processing result is a successful state jump; conversely, if the call processing information does not meet the jump conditions, the job's processing result is a failed state jump. It's important to note that a failed state jump includes two scenarios: call failure and failure to meet the jump conditions.

[0094] Step 108: If the processing result is that the state transition is unsuccessful, add the target state instance to the instance reprocessing queue.

[0095] In the above embodiments, how synchronous state machines and asynchronous state machines call and process jobs and obtain processing results has been explained. In this step, we will further explain how synchronous state machines and asynchronous state machines perform corresponding processing based on the processing results.

[0096] Specifically, when the target state machine's state machine mode is synchronous mode, the instance reprocessing queue includes an instance delay queue and an instance self-circulating queue.

[0097] If the processing result is that the state transition is unsuccessful, the target state instance is added to the instance reprocessing queue, including:

[0098] If the call processing information indicates a call failure, the target state instance is added to the instance delay queue, wherein the instance delay queue is used to store state instances of call failure.

[0099] If the call processing information indicates that the call was successful but the jump condition was not met, the target state instance is added to the instance self-circular queue, wherein the instance self-circular queue is used to store state instances that were successfully called but did not meet the jump condition.

[0100] As mentioned in the steps above, the instance reprocessing queue in the synchronous state machine includes an instance delay queue and an instance self-loop queue. These two queues correspond to two situations where the processing result is an unsuccessful state transition. Specifically, the instance delay queue corresponds to the scenario where the job call fails, and the instance self-loop queue corresponds to the scenario where the job call succeeds but the transition conditions are not met.

[0101] Furthermore, see Figure 3 The delay queue of the synchronous state machine is used to store the state instances (jobs) of failed calls. When the task thread pool calls the job and processes it, it obtains the call processing information. If the call processing information of the job is that the call failed, it means that the job did not succeed in the current call and needs to be called again. Then the job is added to the instance delay queue.

[0102] Correspondingly, the self-circulating queue of the synchronous state machine is used to store state instances where the current call was successful but the jump condition was not met. For example, if the jump condition of a job is "successfully executed 5 times", the current call processing information of the job is successful, but after counting the number of successful executions of the job, it is found that the job has been successfully executed 3 times, which has not yet met the job's jump condition. Therefore, the job needs to be added to the self-circulating queue so that the job can be called and processed here.

[0103] In another specific embodiment provided in this application, the method further includes:

[0104] If the processing result is a successful state transition, the target state instance is added to the state instance queue of the next state machine.

[0105] In a synchronous state machine, if the processing result is a successful state transition, it means that the job has been processed in the current state machine and needs to transition to another task state. In this case, the job is added to the job queue of the next state machine so that the next state machine can process the job accordingly.

[0106] The following explanation uses the case where the target state machine is in asynchronous mode as an example. Specifically, the instance reprocessing queue includes an instance delay queue and an instance self-circulating queue.

[0107] If the processing result is that the state transition is unsuccessful, the target state instance is added to the instance reprocessing queue, including:

[0108] If the call processing information indicates a call failure, the target state instance is added to the instance delay queue, wherein the instance delay queue is used to store state instances of call failure.

[0109] If the call processing information indicates that the call was successful but the jump condition was not met, the target state instance is added to the instance self-circular queue, wherein the instance self-circular queue is used to store state instances that were successfully called but did not meet the jump condition.

[0110] It is important to note that the processing logic for the instance delay queue and instance self-circulating queue in the asynchronous state machine is the same as that in the synchronous state machine. When the external environment returns a call processing information indicating a call failure, the job is added to the instance delay queue; when the external environment returns a call processing information indicating a call success, but the jump condition is not met, the job is added to the instance self-circulating queue. The processing logic for the instance delay queue and instance self-circulating queue in the asynchronous state machine is described in the relevant section on the synchronous state machine, and will not be repeated here.

[0111] It is important to note that there is also an asynchronous thread in the asynchronous state machine, which regenerates a new call information for expired but uncalled jobs in the job cache in real time and puts it into the notification queue to update the original call information so that the external environment can call the job. During this process, the job's tag will be changed to ensure the consistency of external environment calls.

[0112] In practical applications, the asynchronous state machine handles successfully transitioned jobs differently than the synchronous state machine. Specifically, the asynchronous state machine also includes a next state queue, which is used to store state instances that have successfully transitioned.

[0113] The method further includes:

[0114] If the processing result is a successful state transition, the target state instance is added to the next state queue;

[0115] The target state instance is retrieved from the next state queue and added to the state instance queue of the next state machine.

[0116] See Figure 4 ,exist Figure 4 As can be seen, the asynchronous state machine also includes a next state queue, which is used to store jobs whose processing result is a successful state transition. In practical applications, to prevent the asynchronous state machine from being unable to know the completion status of a job after it has been successfully called asynchronously and thus unable to transition the job to the next state in a timely manner, a next state queue is added as a buffer queue. When completion information is received from the external environment, the job is added to the next state queue of the asynchronous state machine, waiting for an asynchronous thread in the asynchronous state machine to retrieve the job from the next state queue and transmit it to the state instance queue of the next state machine.

[0117] Step 110: Add the target state instance in the instance reprocessing queue to the state instance queue, and continue to perform the operation of pulling the target state instance from the state instance queue and performing instance processing on the target state instance based on the state machine pattern of the target state machine.

[0118] The instance reprocessing queue stores jobs that need to be executed again. Therefore, it is necessary to add the jobs in the instance reprocessing queue back to the job queue, and then execute the process of pulling the job from the job queue and processing the job according to the state machine mode of the target state machine.

[0119] It is important to note that the instance reprocessing queue includes an instance deferred queue and an instance self-circulating queue, and the processing methods for these two queues differ. Specifically, adding the target state instance from the instance reprocessing queue to the state instance queue includes:

[0120] After a preset delay period, the target state instance in the instance delay queue is added to the state instance queue; or

[0121] Add the target state instance from the self-circulating queue of instances to the state instance queue.

[0122] For instance delay queues, since these queues store jobs that failed to be executed, in order to ensure that a job can be executed successfully again, it is necessary to wait for a preset delay period before processing the job again. Therefore, when a job is added to the instance delay queue, an asynchronous thread in the target state machine will periodically retrieve the job from the delay queue and place it into the job queue after the preset delay period, so that the job can be called again.

[0123] For instance self-circular queues, an asynchronous thread in the target state machine will retrieve jobs from the instance self-circular queue in real time and place them in the job queue so that the job can be called again.

[0124] The state machine-based task processing method provided in this application obtains a target state instance of a target task and determines a target state machine based on the target state instance. The target state machine includes a state instance queue and an instance reprocessing queue. The method involves adding the target state instance to the state instance queue, retrieving the target state instance from the state instance queue, performing instance processing on the target state instance based on the state machine pattern of the target state machine, and obtaining a processing result. If the processing result indicates that the state transition was unsuccessful, the method adds the target state instance to the instance reprocessing queue. The method then adds the target state instance from the instance reprocessing queue back to the state instance queue and continues to perform the operation of retrieving the target state instance from the state instance queue and performing instance processing on the target state instance based on the state machine pattern of the target state machine.

[0125] The method provided in this application processes the target state instance of the target task through a state instance queue and an instance reprocessing queue. The target state instance is added to the state instance queue and executed through the state instance queue. If the processing result of the target state instance is that the state transition is unsuccessful, the target state instance is added to the instance reprocessing queue and waits to be called for processing again, thus realizing the functions of task failure retry and self-loop.

[0126] Secondly, the state machine in the method provided in this application supports synchronous and asynchronous modes. By performing fine-grained control over each state, a single state machine has stronger control capabilities. At the same time, during asynchronous calls in the external environment, security authentication of external calls is ensured through label marking.

[0127] Finally, this application also provides an intelligent scheduling framework. The intelligent scheduler in the framework supports data storage in various types of databases through an interface. Furthermore, through interaction with the database, it also supports the visualization of state machines.

[0128] The following is in conjunction with the appendix Figure 5 Taking the application of the state machine-based task processing method provided in this application in a data migration scenario as an example, the state machine-based task processing method will be further explained. Figure 5 A schematic diagram of a data migration task process provided in an embodiment of this application is shown.

[0129] In this embodiment, data migration in an erasure coding object storage system is taken as an example. The migration of data to the erasure coding object storage system is logically complex and cannot be completed in a single event; it also requires the assistance of an encoding / decoding computing cluster. A schematic diagram of the data migration task flow in the erasure coding storage system can be found here. Figure 5 The data migration task was divided into 6 steps, each corresponding to a state machine, as follows:

[0130] 1. Prepare: Based on the size of the erasure coding stripe and the information of the external data, perform preparation operations and generate metadata information to record the information of the external files to be migrated.

[0131] 2. allFreeSpace: The erasure coding storage system allocates disk space for this migration stripe and records the disk location to be written.

[0132] 3. waitCoding (Preparing for Encoding): The migration service blocks this task, waiting for the migration task to be actively pulled by the distributed computing nodes. At this time, the distributed computing nodes will perform two operations: First, pull a migration service to execute the encoding and decoding task. Second, execute the encoding and decoding algorithm at the stripe level and calculate the checksum.

[0133] 4. writeData: The migration service concurrently writes the encoded data blocks and check blocks to the disk of the erasure coding storage system.

[0134] 5. writeMeta: In erasure coding storage systems, information related to metadata is recorded, including the disk location of external file storage, and the modification of disk metadata to indicate that data has been written to that location, etc.

[0135] 6. Clear: Clears the metadata information related to migration, completing the migration operation for a single stripe.

[0136] In the entire data migration task process, taking the waitCoding state machine as an example, this state machine is an asynchronous operation, and the waitCoding task will be placed in a position such as... Figure 4The asynchronous state machine shown contains a job cache waiting for calls from distributed computing nodes. The distributed computing nodes will execute the `AcquireTask` (get the flag) task, which is equivalent to calling... Figure 4 The asynchronous call in the process retrieves tasks from the asynchronous state machine. Correspondingly, after the stripe encoding and decoding are completed, the distributed computing nodes call... Figure 4 The notify interface in the asynchronous call drives the flow of the asynchronous state machine corresponding to waitCoding.

[0137] In one specific embodiment of this application, if the call processing information returned by the distributed computing node through the notify interface indicates that the call failed, the job is added to the delay queue. After waiting for a preset time, the job is retrieved from the delay queue and added to the job queue, waiting to be added to the job cache again for call processing.

[0138] In another specific embodiment of this application, if the call processing information returned by the distributed computing node through the notify interface indicates that the execution was successful and the jump condition was not met (which can also be understood as the running state after execution being the same as the current running state), then the job needs to be added to the self-circular queue in the asynchronous state machine, and then the execution thread in the asynchronous state machine takes the job out of the self-circular queue and puts it into the job queue.

[0139] In another specific embodiment of this application, if the call processing information returned by the notify interface of the distributed computing node indicates that the execution was successful and the jump condition is met, the job is added to the next state queue of the asynchronous state machine, and the job is retrieved from the next state queue by another processing thread and added to the job queue corresponding to the writeData state machine.

[0140] The method provided in this application embodiment allows developers to focus on the operation of the migration process logic itself, such as how to record metadata and how to allocate disk space, without having to worry about issues such as how to retry after network anomalies, how to limit the rate, how to jump the state machine, how to coordinate with external systems, and authentication. This greatly simplifies the complexity of development and reduces the development cycle.

[0141] Corresponding to the above-described embodiments of the task processing method based on state machines, this application also provides embodiments of a task processing apparatus based on state machines. Figure 6 A schematic diagram of a state machine-based task processing device according to an embodiment of this application is shown. Figure 6 As shown, the device includes:

[0142] The acquisition module 602 is configured to acquire a target state instance of a target task and determine a target state machine based on the target state instance, wherein the target state machine includes a state instance queue and an instance reprocessing queue.

[0143] The instance queue adding module 604 is configured to add the target state instance to the state instance queue;

[0144] The processing module 606 is configured to pull the target state instance from the state instance queue, perform instance processing on the target state instance based on the state machine pattern of the target state machine, and obtain the processing result;

[0145] The reprocessing queue addition module 608 is configured to add the target state instance to the instance reprocessing queue if the processing result is that the state transition is unsuccessful.

[0146] The reprocessing module 610 is configured to add the target state instance from the instance reprocessing queue to the state instance queue, and continue to perform the operation of pulling the target state instance from the state instance queue and performing instance processing on the target state instance based on the state machine mode of the target state machine.

[0147] Optionally, the acquisition module 602 is further configured to:

[0148] Determine the task state execution type corresponding to the target state instance;

[0149] The state machine mode is determined based on the task state execution type, wherein the state machine mode includes synchronous mode and asynchronous mode;

[0150] The target state machine corresponding to the target state instance is determined based on the state machine pattern, wherein the state machine includes synchronous state machines and asynchronous state machines.

[0151] Optionally, this applies when the target state machine is in synchronous mode;

[0152] The processing module 606 is further configured to:

[0153] The target state instance is invoked and processed through a task thread pool to obtain invocation processing information;

[0154] Obtain the jump condition corresponding to the target state instance;

[0155] If the call processing information does not meet the jump condition, the processing result is determined to be that the state jump was unsuccessful;

[0156] If the call processing information meets the jump conditions, the processing result is determined to be a successful state jump.

[0157] Optionally, the instance reprocessing queue includes an instance delay queue and an instance self-circulating queue;

[0158] The reprocessing queue adding module 608 is further configured to:

[0159] If the call processing information indicates a call failure, the target state instance is added to the instance delay queue, wherein the instance delay queue is used to store state instances of call failure.

[0160] If the call processing information indicates that the call was successful but the jump condition was not met, the target state instance is added to the instance self-circular queue, wherein the instance self-circular queue is used to store state instances that were successfully called but did not meet the jump condition.

[0161] Optionally, the device further includes:

[0162] The state transition success module is configured to add the target state instance to the state instance queue of the next state machine when the processing result is a state transition success.

[0163] Optionally, this applies when the target state machine's state machine mode is asynchronous.

[0164] The processing module 606 is further configured to:

[0165] Add the target state instance to the instance cache corresponding to the target state machine;

[0166] Generate call information corresponding to the target state instance in the notification queue, wherein the call information includes the location information of the target state instance in the instance cache;

[0167] Receive the processing result returned by the asynchronous call based on the call information.

[0168] Optionally, the processing module 606 is further configured to:

[0169] Receive an asynchronous calling program's calling instruction in response to the calling information, wherein the calling instruction includes a task identifier and a task tag;

[0170] Based on the location information, the target state instance is retrieved asynchronously and the target state instance is marked with the task tag.

[0171] Receive the call processing information of the asynchronous call and obtain the jump condition corresponding to the target state instance;

[0172] If the call processing information does not meet the jump condition, the processing result is determined to be that the state jump was unsuccessful;

[0173] If the call processing information meets the jump conditions, the processing result is determined to be a successful state jump.

[0174] Optionally, the instance reprocessing queue includes an instance delay queue and an instance self-circulating queue;

[0175] The reprocessing queue adding module 608 is further configured to:

[0176] If the call processing information indicates a call failure, the target state instance is added to the instance delay queue, wherein the instance delay queue is used to store state instances of call failure.

[0177] If the call processing information indicates that the call was successful but the jump condition was not met, the target state instance is added to the instance self-circular queue, wherein the instance self-circular queue is used to store state instances that were successfully called but did not meet the jump condition.

[0178] Optionally, the asynchronous state machine also includes a next state queue, wherein the next state queue is used to store state instances where a state transition has been successful;

[0179] The device also includes a next queue addition module, configured to:

[0180] If the processing result is a successful state transition, the target state instance is added to the next state queue;

[0181] The target state instance is retrieved from the next state queue and added to the state instance queue of the next state machine.

[0182] Optionally, the reprocessing module 610 is further configured to:

[0183] After a preset delay period, the target state instance in the instance delay queue is added to the state instance queue; or

[0184] Add the target state instance from the self-circulating queue of instances to the state instance queue.

[0185] The state machine-based task processing apparatus provided in this application obtains a target state instance of a target task and determines a target state machine based on the target state instance. The target state machine includes a state instance queue and an instance reprocessing queue. The apparatus adds the target state instance to the state instance queue, retrieves the target state instance from the state instance queue, performs instance processing on the target state instance based on the state machine pattern of the target state machine, and obtains a processing result. If the processing result indicates that the state transition was unsuccessful, the apparatus adds the target state instance to the instance reprocessing queue. The apparatus then adds the target state instance from the instance reprocessing queue back to the state instance queue and continues to perform the operation of retrieving the target state instance from the state instance queue and performing instance processing on the target state instance based on the state machine pattern of the target state machine.

[0186] The apparatus provided in this application processes the target state instance of a target task through a state instance queue and an instance reprocessing queue. The target state instance is added to the state instance queue and executed through the state instance queue. If the processing result of the target state instance is that the state transition is unsuccessful, the target state instance is added to the instance reprocessing queue and waits to be called for processing again, thus realizing the functions of task failure retry and self-loop.

[0187] Secondly, the state machine in the device provided in this application supports synchronous and asynchronous modes. By performing fine-grained control on each state, the single state machine has stronger control capabilities. At the same time, during asynchronous calls in the external environment, the security authentication of external calls is ensured through label marking.

[0188] Finally, this application also provides an intelligent scheduling framework. The intelligent scheduler in the framework supports data storage in various types of databases through an interface. Furthermore, through interaction with the database, it also supports the visualization of state machines.

[0189] The above is a schematic scheme of a state machine-based task processing device according to this embodiment. It should be noted that the technical solution of this state machine-based task processing device and the technical solution of the state machine-based task processing method described above belong to the same concept. For details not described in detail in the technical solution of the state machine-based task processing device, please refer to the description of the technical solution of the state machine-based task processing method described above.

[0190] Figure 7A structural block diagram of a computing device 700 according to an embodiment of this application is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0191] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0192] In one embodiment of this application, the aforementioned components of the computing device 700 and Figure 7 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 7 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0193] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 700 can also be a mobile or stationary server.

[0194] The processor 720 implements the state machine-based task processing method when executing the computer instructions.

[0195] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the state machine-based task processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the state machine-based task processing method described above.

[0196] An embodiment of this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the state machine-based task processing method described above.

[0197] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the state machine-based task processing method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the state machine-based task processing method described above.

[0198] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0199] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0200] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0201] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0202] The preferred embodiments disclosed above are merely illustrative of this application. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this application. These embodiments are selected and specifically described in this application to better explain the principles and practical applications of this application, thereby enabling those skilled in the art to better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A task processing method based on a state machine, characterized in that, include: Obtain the target state instance of the target task, and determine the target state machine based on the target state instance, wherein the target state instance is the current subtask corresponding to the target task, and the execution instance in the current subtask state; the target state machine includes a state instance queue and an instance reprocessing queue. Add the target state instance to the state instance queue; The target state instance is retrieved from the state instance queue, and the target state instance is processed based on the state machine pattern of the target state machine to obtain the processing result. If the processing result is that the state transition is unsuccessful, the target state instance is added to the instance reprocessing queue; The target state instance in the instance reprocessing queue is added to the state instance queue, and the operation of pulling the target state instance from the state instance queue and performing instance processing on the target state instance based on the state machine pattern of the target state machine continues.

2. The method as described in claim 1, characterized in that, Determining the target state machine based on the target state instance includes: Determine the task state execution type corresponding to the target state instance; The state machine mode is determined based on the task state execution type, wherein the state machine mode includes synchronous mode and asynchronous mode; The target state machine corresponding to the target state instance is determined based on the state machine pattern, wherein the state machine includes synchronous state machines and asynchronous state machines.

3. The method as described in claim 2, characterized in that, When the state machine mode of the target state machine is synchronous mode; The target state instance is retrieved from the state instance queue, and instance processing is performed on the target state instance based on the state machine pattern of the target state machine to obtain the processing result, including: The target state instance is invoked and processed through a task thread pool to obtain invocation processing information; Obtain the jump condition corresponding to the target state instance; If the call processing information does not meet the jump condition, the processing result is determined to be that the state jump was unsuccessful; If the call processing information meets the jump conditions, the processing result is determined to be a successful state jump.

4. The method as described in claim 3, characterized in that, The instance reprocessing queue includes an instance delay queue and an instance self-circulating queue; If the processing result is that the state transition is unsuccessful, the target state instance is added to the instance reprocessing queue, including: If the call processing information indicates a call failure, the target state instance is added to the instance delay queue, wherein the instance delay queue is used to store state instances of call failure. If the call processing information indicates that the call was successful but the jump condition was not met, the target state instance is added to the instance self-circular queue, wherein the instance self-circular queue is used to store state instances that were successfully called but did not meet the jump condition.

5. The method as described in claim 3, characterized in that, The method further includes: If the processing result is a successful state transition, the target state instance is added to the state instance queue of the next state machine.

6. The method as described in claim 2, characterized in that, When the target state machine is in asynchronous mode; The target state instance is retrieved from the state instance queue, and instance processing is performed on the target state instance based on the state machine pattern of the target state machine to obtain the processing result, including: Add the target state instance to the instance cache corresponding to the target state machine; Generate call information corresponding to the target state instance in the notification queue, wherein the call information includes the location information of the target state instance in the instance cache; Receive the processing result returned by the asynchronous call based on the call information.

7. The method as described in claim 6, characterized in that, Receiving the processing result returned by the asynchronous call based on the call information includes: Receive an asynchronous calling program's calling instruction in response to the calling information, wherein the calling instruction includes a task identifier and a task tag; Based on the location information, the target state instance is retrieved asynchronously and the target state instance is marked with the task tag. Receive the call processing information of the asynchronous call and obtain the jump condition corresponding to the target state instance; If the call processing information does not meet the jump condition, the processing result is determined to be that the state jump was unsuccessful; If the call processing information meets the jump conditions, the processing result is determined to be a successful state jump.

8. The method as described in claim 7, characterized in that, The instance reprocessing queue includes an instance delay queue and an instance self-circulating queue; If the processing result is that the state transition is unsuccessful, the target state instance is added to the instance reprocessing queue, including: If the call processing information indicates a call failure, the target state instance is added to the instance delay queue, wherein the instance delay queue is used to store state instances of call failure. If the call processing information indicates that the call was successful but the jump condition was not met, the target state instance is added to the instance self-circular queue, wherein the instance self-circular queue is used to store state instances that were successfully called but did not meet the jump condition.

9. The method as described in claim 7, characterized in that, The asynchronous state machine also includes a next state queue, which is used to store state instances where a state transition was successful. The method further includes: If the processing result is a successful state transition, the target state instance is added to the next state queue; The target state instance is retrieved from the next state queue and added to the state instance queue of the next state machine.

10. The method as described in claim 4 or 8, characterized in that, Adding the target state instance from the instance reprocessing queue to the state instance queue includes: After a preset delay period, the target state instance in the instance delay queue is added to the state instance queue; or Add the target state instance from the self-circulating queue of instances to the state instance queue.

11. A task processing device based on a state machine, characterized in that, include: The acquisition module is configured to acquire a target state instance of a target task and determine a target state machine based on the target state instance, wherein the target state instance is the current subtask corresponding to the target task and the execution instance in the current subtask state; the target state machine includes a state instance queue and an instance reprocessing queue. The instance queue addition module is configured to add the target state instance to the state instance queue. The processing module is configured to pull the target state instance from the state instance queue, perform instance processing on the target state instance based on the state machine pattern of the target state machine, and obtain the processing result; The reprocessing queue addition module is configured to add the target state instance to the instance reprocessing queue if the processing result is that the state transition is unsuccessful. The reprocessing module is configured to add the target state instance from the instance reprocessing queue to the state instance queue, and continue to perform the operation of pulling the target state instance from the state instance queue and performing instance processing on the target state instance based on the state machine pattern of the target state machine.

12. A computing device, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the computer instructions, it implements the steps of the method according to any one of claims 1-10.

13. A computer-readable storage medium storing computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the method according to any one of claims 1-10.

14. A computer program product, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-10.

Images