Asynchronous task processing method and device and computer readable storage medium

Abstract

The application relates to the technical field of data processing, and discloses an asynchronous task processing method, which comprises the following steps: receiving an asynchronous task, declaring the asynchronous task through an annotation, obtaining a source asynchronous task, and storing the source asynchronous task in a database after converting the source asynchronous task into a database table; assigning a machine address to the source asynchronous task through a public registration center, and establishing a sharding item of the source asynchronous task through a pre-constructed asynchronous framework, wherein the machine address and an index in the sharding item are in one-to-one correspondence; obtaining the corresponding index in the sharding item from the public registration center through the machine address assigned by the source asynchronous task, obtaining a corresponding sharding task according to the corresponding index, and executing the corresponding sharding task through multi-thread concurrent execution. The application further discloses an asynchronous task processing device, an electronic device and a computer readable storage medium. The application can realize the processing of asynchronous tasks.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an asynchronous task processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Asynchronous tasks are a concept distinct from synchronous tasks. Synchronous tasks refer to requests that, when executed, cannot return until a response is received; the request must block until the response is available before proceeding to the next step. Asynchronous tasks, on the other hand, do not require an immediate response when executing a task; the operation is processed only when a response is received. With the continuous development of information technology and the expansion of demands, enterprises urgently need to efficiently and quickly handle various asynchronous task requests. Currently, traditional asynchronous frameworks face the following three problems when handling asynchronous tasks:

[0003] First, traditional asynchronous frameworks suffer from the single-machine problem of asynchronous task processing. This is because the processing logic for the same asynchronous task is the same. If the same logic repeatedly processes the same data, it may cause unpredictable risks.

[0004] Secondly, traditional asynchronous frameworks suffer from uncontrollable scheduling and data duplication issues.

[0005] Third, there are no follow-up operations after the traditional asynchronous task is completed, so it is impossible to predict whether the asynchronous task will succeed or fail. If there are hidden bugs in the program, they can only be discovered by the business side raising the issue, which poses a certain risk to the production environment. Summary of the Invention

[0006] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for asynchronous task processing. Its main purpose is to help users solve the problems of single-machine operation and data duplication during asynchronous task processing, as well as the problem of not being able to know whether the asynchronous task was successful after processing.

[0007] To achieve the above objectives, the present invention provides a storage method for asynchronous task processing, comprising:

[0008] Receive asynchronous tasks, declare the asynchronous tasks through annotations to obtain source asynchronous tasks, and then store the source asynchronous tasks in the database after converting them into database tables.

[0009] The source asynchronous task is assigned a machine address through a public registry center, and a sharding item for the source asynchronous task is established through a pre-built asynchronous framework, wherein the machine address corresponds one-to-one with the index in the sharding item;

[0010] The corresponding index in the sharding item is obtained from the public registry center through the machine address allocated by the source asynchronous task, and the corresponding sharding task is obtained according to the corresponding index. The corresponding sharding task is then executed concurrently using multi-threading.

[0011] Optionally, before the concurrent execution of the corresponding sharded task using multi-threading, the following is included:

[0012] Create a node lock on the sharded item in the source asynchronous task to prevent the source asynchronous task from being executed repeatedly.

[0013] Optionally, creating a node lock on the sharded item in the source asynchronous task to prevent repeated execution of the source asynchronous task includes:

[0014] Generate the lock name of the node lock in the sharding item in the source asynchronous task;

[0015] The lock name is used as a parameter to call back to the public registry center to obtain the execution order sequence of the node locks;

[0016] The node locks for the shard items are generated according to the execution order sequence of the node locks.

[0017] Optionally, after the concurrent execution of the corresponding sharded task using multi-threading, the following is included:

[0018] Record information during the execution of the source asynchronous task's sharded tasks, and re-execute the sharded tasks that fail through a pre-defined retry mechanism.

[0019] Optionally, the re-execution of the failed sharded task through a pre-set retry mechanism includes:

[0020] A preset integer variable sum is used to retry the execution of the failed shard task and record the retry count sum+1. The integer variable sum is updated using the value of the retry count sum+1 until the value of the integer variable sum is greater than the preset maximum number of retries, thus completing the re-execution of the failed shard task.

[0021] Optionally, the step of converting the source asynchronous task into a database table and storing it in the database includes:

[0022] The source asynchronous task is converted into a database table by creating fields using SQL statements and matching the data in the source asynchronous task according to the corresponding fields.

[0023] Optionally, the step of creating sharded items for the source asynchronous task using a pre-built asynchronous framework includes:

[0024] The source asynchronous task is sharded using a preset sharding algorithm to create sharded items, wherein the sharding algorithm includes:

[0025] FI=(to_big_integer(to_byte, 32)%serviceCount)+1;

[0026] Where FI represents the shard entry to be created, to_byte refers to the node number, to_big_integer represents the maximum value between the node number and the number 32 converted to an integer, and serviceCount represents the number of times it is necessary to create a shard entry.

[0027] To address the above problems, the present invention also provides an asynchronous task processing apparatus, the apparatus comprising:

[0028] The declaration and conversion module is used to receive asynchronous tasks, declare the asynchronous tasks through annotations to obtain source asynchronous tasks, and convert the source asynchronous tasks into database tables and store them in the database.

[0029] The allocation and creation module is used to allocate machine addresses to the source asynchronous task through a public registry center, and to create shards for the source asynchronous task through a pre-built asynchronous framework, wherein the machine address corresponds one-to-one with the index in the shard.

[0030] The acquisition and execution module is used to obtain the corresponding index in the shard item from the public registry center through the machine address allocated by the source asynchronous task, and obtain the corresponding shard task according to the corresponding index, and execute the corresponding shard task concurrently using multi-threading.

[0031] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0032] Memory, storing at least one instruction; and

[0033] The processor executes instructions stored in the memory to implement the asynchronous task processing method described in any one of the above.

[0034] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the asynchronous task processing method described in any one of the preceding embodiments.

[0035] In this embodiment of the invention, asynchronous tasks are declared and transformed using annotations and then stored in a database. A public registry center allocates machine addresses to the asynchronous tasks, and a pre-built asynchronous framework is used to create sharded entries for the source asynchronous tasks. This ensures that asynchronous tasks do not operate on a single machine during processing. The machine address is used to retrieve the corresponding sharded task from the public registry center. By executing the corresponding sharded task concurrently using multiple threads, the processing status of asynchronous tasks can be viewed in real time, helping users to efficiently process asynchronous tasks.

[0036] Furthermore, in this embodiment of the invention, before executing the asynchronous task, a node lock is created for the sharded items in the asynchronous task to ensure that data is not executed repeatedly in the asynchronous task.

[0037] Furthermore, in this embodiment of the invention, after executing the corresponding sharded task, information during the execution of the source asynchronous task is recorded to determine whether the asynchronous task was executed successfully. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating an asynchronous task processing method according to an embodiment of the present invention.

[0039] Figure 2 A schematic diagram of the modules of an asynchronous task processing method provided in an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of the internal structure of an electronic device for an asynchronous task processing method provided in an embodiment of the present invention;

[0041] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0042] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0043] This invention provides a method for asynchronous task processing. (See attached document.) Figure 1 The diagram shown is a flowchart illustrating an asynchronous task processing method according to an embodiment of the present invention. This method can be executed by a device, which can be implemented in software and / or hardware.

[0044] In this embodiment, the method for asynchronous task processing includes:

[0045] S1. Receive asynchronous tasks, declare the asynchronous tasks through annotations to obtain source asynchronous tasks, and store the source asynchronous tasks in the database.

[0046] In a preferred embodiment of the present invention, the annotation refers to a special marker in the program code, and the format of the annotation is @annotation_name. The special marker can be read during program code compilation, class loading, and runtime, and corresponding processing can be performed. This embodiment of the present invention uses annotations to embed supplementary information without changing the original program code and logic, achieving a declarative function. Specifically, this embodiment of the present invention defines the annotation type upon its first appearance, using the @interface keyword (explaining the @interface keyword) to annotate the asynchronous task, thus obtaining the source asynchronous task. The @interface keyword is equivalent to an annotation class, used to define the type of data.

[0047] Preferably, in this embodiment of the invention, the source asynchronous task is converted into a database table and then stored in a database. The database table is a regular relational database table used to store basic information about the source asynchronous task. Preferably, the database is an Oracle database.

[0048] In detail, the specific implementation method for converting the source asynchronous task into a database table includes:

[0049] a. Create the fields of the source asynchronous task using SQL statements, wherein the fields include DD_TIME, STSTE, TYPE, COUNT, etc.;

[0050] b. Match the data in the source asynchronous task according to the corresponding fields. For example, store the data as a JSON string, indicating the value of each data item by index increment, such as [1:80101, 2:802, 3:905|906]. The right side of the colon is the shard value, and the left side of the colon is the shard index, which corresponds to the machine. The machine can be the device that will execute the source asynchronous task.

[0051] S2. Assign machine addresses to the source asynchronous task through a public registry center, and establish sharding items for the source asynchronous task through a pre-built asynchronous framework, wherein the machine address corresponds one-to-one with the index in the sharding item.

[0052] In a preferred embodiment of the present invention, the public registry center is ZooKeeper (ZK). The present invention sends the source asynchronous task to the ZK for registration, and the ZK allocates the source asynchronous task to the corresponding machine to obtain the corresponding machine address. Preferably, the present invention allocates the corresponding machine by calling methods in a local program and / or accessing other servers. Here, the machine address can be understood in the present invention as the program that processes the source asynchronous task, wherein the program can be an application method or a physical address, i.e., an IP address. Here, ZK is similar to an object, which broadly refers to a meaningful region in memory.

[0053] Preferably, the pre-built asynchronous framework described in this embodiment of the invention is used to realize the approval data connection between two systems A and B. For example, system A is an approval system of a certain city, and system B is an approval system of a certain government unit in the same city. Interface C is used to provide system B with the final opinion after approval by system B and push it to system A. Herein, the interface described in this invention is a web service, i.e., a server.

[0054] The pre-built asynchronous framework also supports client-side configuration parameter modification via a graphical interface, allowing for real-time modification of asynchronous task parameters. For example, each asynchronous task is used as the smallest unit of modification. A written SQL script is executed programmatically to modify the task's call time, task information, and other configurations. A built-in listener is included; once a change in configuration information is detected, the corresponding client configuration information is immediately refreshed, enabling real-time monitoring and modification. The SQL script can be a DML statement or a stored procedure, such as: `CREATE OR REPLACEPROCEDURE testname(argument1 TYPE1, ...) AS BEGIN ... END testname`.

[0055] Preferably, in this embodiment of the invention, the source asynchronous task is sharded using the sharding algorithm of the asynchronous framework to establish sharded items. More preferably, this invention associates the sharded items with task units in system A and system B, where the task unit refers to the carrier of the source asynchronous task to be executed.

[0056] The sharding algorithm includes:

[0057] FI=(to_big_integer(to_byte, 32)%serviceCount)+1

[0058] Wherein, FI represents the shard entry to be created, to_byte refers to the node number, to_big_integer represents the maximum value between the node number and the number 32 converted to an integer, and serviceCount represents the number of times it is necessary to create a shard entry. The to_byte (node ​​number) is a unique ID that identifies the source asynchronous task. The node number refers to the index created when the source asynchronous task is declared.

[0059] Furthermore, in this embodiment of the invention, based on the task information in the task table saved at the time of declaration, the corresponding machine address of the task is queried and index entries are allocated according to the format of shard item index_method name or shard item_machine address, so as to achieve a one-to-one correspondence between the index in the shard item and the machine address.

[0060] S3. Obtain the corresponding index in the sharding item from the public registry center through the machine address allocated by the source asynchronous task, and obtain the corresponding sharding task according to the corresponding index, and execute the corresponding sharding task concurrently using multi-threading.

[0061] In a preferred embodiment of the present invention, when executing the source asynchronous task, the corresponding index in the shard item is obtained from the ZooKeeper (ZK) through the machine address allocated to the source asynchronous task, and the corresponding shard task is obtained according to the corresponding index. The corresponding shard task is then executed concurrently using multithreading. Here, a thread is the smallest unit of computation scheduling that an operating system can perform; it is contained within a program (process) and is the actual unit of operation within the program. A thread refers to a single sequential flow of control in a program, usually understood as one thread executing one task. Multithreading refers to the technology of enabling multiple threads to execute concurrently, i.e., executing multiple shards simultaneously. Although the execution processes of threads do not affect each other, they may operate on the same data. Therefore, when executing shard tasks, the present invention avoids data conflicts during data filtering in shard tasks by adding the shard item as a WHERE condition in the SQL statement.

[0062] Furthermore, a preferred embodiment of the present invention further includes creating a node lock on the shards in the source asynchronous task before executing the source asynchronous task, so as to avoid repeated execution of the source asynchronous task.

[0063] In detail, the specific implementation steps for creating node locks on sharded items in the source asynchronous task include:

[0064] Before the source asynchronous task starts, the asynchronous framework pre-built in S2 above will automatically generate a lock name of node lock named lock_name1_ in the shard item of the source asynchronous task. Since the policy number is unique, the lock name of the policy number is also unique. Further, this invention uses the lock name as a parameter to call back to the aforementioned public registry center, thereby obtaining the execution order sequence (List) of the node locks. Based on the execution order sequence of the node locks, the node lock of the shard item is generated. The List is similar to an array, storing a set of sorted lock names. Before execution, the source asynchronous task verifies whether its lock name is first in the List collection. If the lock name is first, it indicates that the client of the shard item is obtaining data in real time. Therefore, when the interval between calls to the shard item is short, for example, the client obtains data every 5 seconds, and the execution time is 6 seconds, there may be a risk of repeated execution, thus avoiding the potential for data conflict.

[0065] Furthermore, embodiments of the present invention also include recording information during the execution of the source asynchronous task after executing the corresponding sharding task, and for sharding tasks that fail to execute, initiating a retry mechanism of a preset number of times, and re-executing the sharding task that failed to execute through the preset number of retry mechanisms.

[0066] Preferably, the present invention records information during the execution of each source asynchronous task through a log in the pre-built asynchronous framework. This information includes the execution result, runtime, and reason for failure. Specifically, a new table, ZX_RESULT, is created in the database, with fields including result, info, and task_id. When the source asynchronous task executes successfully, the task units in the pre-built asynchronous framework consume data normally. When the source asynchronous task fails, an exception is caught using a try / catch mechanism, and a retry mechanism is initiated within the catch mechanism to re-execute the source asynchronous task. The exception refers to errors that occur during program execution, such as functional malfunctions or connection interruptions. These errors include errors in the program itself, network errors, and database connection errors.

[0067] Preferably, the retry mechanism described in this embodiment of the invention is a limited number of retries. Specifically, an integer variable `sum` is preset in the `catch` mechanism. The failed sharded task is retried, and the retry count `sum+1` is recorded. The integer variable `sum` is updated using the value of `sum+1` until the value of `sum` exceeds the preset maximum number of retries, thus completing the re-execution of the failed sharded task. Preferably, the preset maximum number of retries in this invention is 5. This retry mechanism effectively improves the success rate of task execution and subsequent task operations, which is beneficial for maintaining subsequent task analysis, result querying, and data backup.

[0068] like Figure 2 The diagram shown is a functional block diagram of the asynchronous task processing device of the present invention.

[0069] The asynchronous task processing device 100 of this invention can be installed in an electronic device. Depending on the functions implemented, the asynchronous task processing device 100 may include a declaration and conversion module 101, an allocation and creation module 102, and an acquisition and execution module 103. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0070] In this embodiment, the functions of each module / unit are as follows:

[0071] The declaration and conversion module 101 is used to receive asynchronous tasks, declare the asynchronous tasks through annotations to obtain source asynchronous tasks, and convert the source asynchronous tasks into database tables and store them in the database.

[0072] The allocation and creation module 102 is used to allocate machine addresses to the source asynchronous task through a public registry center, and to establish sharding items for the source asynchronous task through a pre-built asynchronous framework, wherein the machine address corresponds one-to-one with the index in the sharding item;

[0073] The acquisition and execution module 103 is used to obtain the corresponding index in the shard item from the public registry center through the machine address allocated by the source asynchronous task, and obtain the corresponding shard task according to the corresponding index, and execute the corresponding shard task concurrently using multi-threading.

[0074] In detail, the specific implementation steps of each module of the storage device for asynchronous task processing are as follows:

[0075] The declaration and conversion module 101 receives asynchronous tasks, declares the asynchronous tasks through annotations to obtain source asynchronous tasks, and converts the source asynchronous tasks into database tables and stores them in the database.

[0076] In a preferred embodiment of the present invention, the annotation refers to a special marker in the program code, and the format of the annotation is @annotation_name. The special marker can be read during program code compilation, class loading, and runtime, and corresponding processing can be performed. This embodiment of the present invention uses annotations to embed supplementary information without changing the original program code and logic, achieving a declarative function. Specifically, this embodiment of the present invention defines the annotation type upon its first appearance, using the @interface keyword (explaining the @interface keyword) to annotate the asynchronous task, thus obtaining the source asynchronous task. The @interface keyword is equivalent to an annotation class, used to define the type of data.

[0077] Preferably, in this embodiment of the invention, the source asynchronous task is converted into a database table and then stored in a database. The database table is a regular relational database table used to store basic information about the source asynchronous task. Preferably, the database is an Oracle database.

[0078] In detail, the specific implementation method for converting the source asynchronous task into a database table includes:

[0079] a. Create the fields of the source asynchronous task using SQL statements, wherein the fields include DD_TIME, STSTE, TYPE, COUNT, etc.;

[0080] b. Match the data in the source asynchronous task according to the corresponding fields. For example, store the data as a JSON string, indicating the value of each data item by index increment, such as [1:80101, 2:802, 3:905|906]. The right side of the colon is the shard value, and the left side of the colon is the shard index, which corresponds to the machine. The machine can be the device that will execute the source asynchronous task.

[0081] The allocation and creation module 102 allocates machine addresses to the source asynchronous task through a public registry center, and establishes shard entries for the source asynchronous task through a pre-built asynchronous framework, wherein the machine address corresponds one-to-one with the index in the shard entry.

[0082] In a preferred embodiment of the present invention, the public registry center is ZooKeeper (ZK). The present invention sends the source asynchronous task to the ZK for registration, and the ZK allocates the source asynchronous task to the corresponding machine to obtain the corresponding machine address. Preferably, the present invention allocates the corresponding machine by calling methods in a local program and / or accessing other servers. Here, the machine address can be understood in the present invention as the program that processes the source asynchronous task, wherein the program can be an application method or a physical address, i.e., an IP address. Here, ZK is similar to an object, which broadly refers to a meaningful region in memory.

[0083] Preferably, the pre-built asynchronous framework described in this embodiment of the invention is used to realize the approval data connection between two systems A and B. For example, system A is an approval system of a certain city, and system B is an approval system of a certain government unit in the same city. Interface C is used to provide system B with the final opinion after approval by system B and push it to system A. Herein, the interface described in this invention is a web service, i.e., a server.

[0084] The pre-built asynchronous framework also supports client-side configuration parameter modification via a graphical interface, allowing for real-time modification of asynchronous task parameters. For example, each asynchronous task is used as the smallest unit of modification. A written SQL script is executed programmatically to modify the task's call time, task information, and other configurations. A built-in listener is included; once a change in configuration information is detected, the corresponding client configuration information is immediately refreshed, enabling real-time monitoring and modification. The SQL script can be a DML statement or a stored procedure, such as: `CREATE OR REPLACEPROCEDURE testname(argument1 TYPE1, ...) AS BEGIN ... END testname`.

[0085] Preferably, in this embodiment of the invention, the source asynchronous task is sharded using the sharding algorithm of the asynchronous framework to establish sharded items. More preferably, this invention associates the sharded items with task units in system A and system B, where the task unit refers to the carrier of the source asynchronous task to be executed.

[0086] The sharding algorithm includes:

[0087] FI=(to_big_integer(to_byte, 32)%serviceCount)+1

[0088] Wherein, FI represents the shard entry to be created, to_byte refers to the node number, to_big_integer represents the maximum value between the node number and the number 32 converted to an integer, and serviceCount represents the number of times it is necessary to create a shard entry. The to_byte (node ​​number) is a unique ID that identifies the source asynchronous task. The node number refers to the index created when the source asynchronous task is declared.

[0089] Furthermore, in this embodiment of the invention, based on the task information in the task table saved at the time of declaration, the corresponding machine address of the task is queried and index entries are allocated according to the format of shard item index_method name or shard item_machine address, so as to achieve a one-to-one correspondence between the index in the shard item and the machine address.

[0090] The acquisition and execution module 103 obtains the corresponding index in the shard item from the public registry center through the machine address allocated by the source asynchronous task, and obtains the corresponding shard task according to the corresponding index, and executes the corresponding shard task concurrently using multi-threading.

[0091] In a preferred embodiment of the present invention, when executing the source asynchronous task, the corresponding index in the shard item is obtained from the ZooKeeper (ZK) through the machine address allocated to the source asynchronous task, and the corresponding shard task is obtained according to the corresponding index. The corresponding shard task is then executed concurrently using multithreading. Here, a thread is the smallest unit of computation scheduling that an operating system can perform; it is contained within a program (process) and is the actual unit of operation within the program. A thread refers to a single sequential flow of control in a program, usually understood as one thread executing one task. Multithreading refers to the technology of enabling multiple threads to execute concurrently, i.e., executing multiple shards simultaneously. Although the execution processes of threads do not affect each other, they may operate on the same data. Therefore, when executing shard tasks, the present invention avoids data conflicts during data filtering in shard tasks by adding the shard item as a WHERE condition in the SQL statement.

[0092] Furthermore, a preferred embodiment of the present invention further includes creating a node lock on the shards in the source asynchronous task before executing the source asynchronous task, so as to avoid repeated execution of the source asynchronous task.

[0093] In detail, the specific implementation steps for creating node locks on sharded items in the source asynchronous task include:

[0094] Before the source asynchronous task starts, the asynchronous framework pre-built in S2 above will automatically generate a lock name of node lock named lock_name1_ in the shard item of the source asynchronous task. Since the policy number is unique, the lock name of the policy number is also unique. Further, this invention uses the lock name as a parameter to call back to the aforementioned public registry center, thereby obtaining the execution order sequence (List) of the node locks. Based on the execution order sequence of the node locks, the node lock of the shard item is generated. The List is similar to an array, storing a set of sorted lock names. Before execution, the source asynchronous task verifies whether its lock name is first in the List collection. If the lock name is first, it indicates that the client of the shard item is obtaining data in real time. Therefore, when the interval between calls to the shard item is short, for example, the client obtains data every 5 seconds, and the execution time is 6 seconds, there may be a risk of repeated execution, thus avoiding the potential for data conflict.

[0095] Furthermore, embodiments of the present invention also include recording information during the execution of the source asynchronous task after executing the corresponding sharding task, and for sharding tasks that fail to execute, initiating a retry mechanism of a preset number of times, and re-executing the sharding task that failed to execute through the preset number of retry mechanisms.

[0096] Preferably, the present invention records information during the execution of each source asynchronous task through a log in the pre-built asynchronous framework. This information includes the execution result, runtime, and reason for failure. Specifically, a new table, ZX_RESULT, is created in the database, with fields including result, info, and task_id. When the source asynchronous task executes successfully, the task units in the pre-built asynchronous framework consume data normally. When the source asynchronous task fails, an exception is caught using a try / catch mechanism, and a retry mechanism is initiated within the catch mechanism to re-execute the source asynchronous task. The exception refers to errors that occur during program execution, such as functional malfunctions or connection interruptions. These errors include errors in the program itself, network errors, and database connection errors.

[0097] Preferably, the retry mechanism described in this embodiment of the invention is a limited number of retries. Specifically, an integer variable `sum` is preset in the `catch` mechanism. The failed shard task is retried, and the retry count `sum+1` is recorded. The integer variable `sum` is updated using the value of `sum+1` until the value of `sum` exceeds the preset maximum number of retries, thus completing the re-execution of the failed shard task. Preferably, the preset maximum number of retries in this invention is 5. This retry mechanism effectively improves the success rate of task execution and subsequent task operations, which is beneficial for maintaining subsequent task analysis, result querying, and data backup.

[0098] like Figure 3 The diagram shown is a structural schematic of an electronic device that implements the asynchronous task processing method of the present invention.

[0099] The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program, such as an asynchronous task handler 12, stored in the memory 11 and executable on the processor 10.

[0100] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 11 can include both internal and external storage units of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of asynchronous task processing programs, but also to temporarily store data that has been output or will be output.

[0101] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules (e.g., asynchronous task processing programs) stored in the memory 11, and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0102] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0103] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0104] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0105] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0106] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0107] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0108] The asynchronous task processing program 12 stored in the memory 11 of the electronic device 1 is a combination of multiple instructions, which, when run in the processor 10, can achieve the following:

[0109] Receive asynchronous tasks, declare the asynchronous tasks through annotations to obtain source asynchronous tasks, and then store the source asynchronous tasks in the database after converting them into database tables.

[0110] The source asynchronous task is assigned a machine address through a public registry center, and a sharding item for the source asynchronous task is established through a pre-built asynchronous framework, wherein the machine address corresponds one-to-one with the index in the sharding item;

[0111] The corresponding index in the sharding item is obtained from the public registry center through the machine address allocated by the source asynchronous task, and the corresponding sharding task is obtained according to the corresponding index. The corresponding sharding task is then executed concurrently using multi-threading.

[0112] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0113] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0114] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0115] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0116] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0117] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0118] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0119] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An asynchronous task processing method, characterized in that, The method includes: Receive asynchronous tasks, declare the asynchronous tasks through annotations to obtain source asynchronous tasks, and then store the source asynchronous tasks in the database after converting them into database tables. The source asynchronous task is assigned a machine address through a public registry center, and a pre-built asynchronous framework is used to shard the source asynchronous task using a preset sharding algorithm to create shard entries for the source asynchronous task. These shard entries are associated with the carrier of the machine address executing the source asynchronous task. Based on the task information in the database table, the corresponding machine address of the task is queried, and index entries are assigned according to the format of shard entry index_method name or shard entry_machine address, achieving a one-to-one correspondence between the machine address and the index in the shard entry. The sharding algorithm includes: FI=(to_big_integer(to_byte,32)%serviceCount)+1; Where FI represents the shard entry to be created, to_byte refers to the node number, to_big_integer represents the maximum value between the node number and the number 32 converted to an integer, and serviceCount represents the number of times it is necessary to create a shard entry; The system obtains the corresponding index in the shard item from the public registry center using the machine address allocated by the source asynchronous task, and obtains the corresponding shard task based on the corresponding index. It generates the lock name of the node lock in the shard item in the source asynchronous task, calls back the lock name as a parameter to the public registry center, obtains the execution order sequence of the node lock, generates the node lock of the shard item based on the execution order sequence of the node lock, and executes the corresponding shard task concurrently using multi-threading.

2. The asynchronous task processing method as described in claim 1, characterized in that, After the method utilizes multi-threaded concurrent execution of the corresponding sharded tasks, it further includes: Record information during the execution of the source asynchronous task's sharded tasks, and re-execute the sharded tasks that fail through a pre-defined retry mechanism.

3. The asynchronous task processing method as described in claim 2, characterized in that, The retry mechanism for the failed sharded task, which involves retrying a preset number of times, includes: A preset integer variable sum is used to retry the execution of the failed shard task and record the retry count sum+1. The integer variable sum is updated using the value of the retry count sum+1 until the value of the integer variable sum is greater than the preset maximum number of retries, thus completing the re-execution of the failed shard task.

4. The asynchronous task processing method as described in any one of claims 1 to 3, characterized in that, The step of converting the source asynchronous task into a database table and storing it in the database includes: The source asynchronous task is created using SQL statements, and the data in the source asynchronous task is matched according to the corresponding fields, thereby converting the source asynchronous task into the database table and storing the database table in the database.

5. An asynchronous task processing device, characterized in that, The device includes: The declaration and conversion module is used to receive asynchronous tasks, declare the asynchronous tasks through annotations to obtain source asynchronous tasks, and convert the source asynchronous tasks into database tables and store them in the database. The allocation and creation module is used to allocate machine addresses to the source asynchronous tasks through a public registry center, and to create shard items for the source asynchronous tasks by sharding the source asynchronous tasks using a preset sharding algorithm through a pre-built asynchronous framework. The shard items are associated with the carriers where the machine addresses executing the source asynchronous tasks reside. Based on the task information in the database table, the module queries the corresponding machine addresses of the tasks and allocates index items according to the format of shard item index_method name or shard item_machine address, achieving a one-to-one correspondence between the machine addresses and the indexes in the shard items. The sharding algorithm includes: FI = (to_big_integer(to_byte, 32)%serviceCount) + 1; where FI represents the created shard item, to_byte refers to the node number, to_big_integer represents the maximum value between the node number and the integer 32, and serviceCount represents the number of times a shard item needs to be created. The acquisition and execution module is used to obtain the corresponding index in the shard item from the public registry center through the machine address allocated by the source asynchronous task, and obtain the corresponding shard task according to the corresponding index. It generates the lock name of the node lock in the shard item in the source asynchronous task, calls back the lock name as a parameter to the public registry center to obtain the execution order sequence of the node lock, generates the node lock of the shard item according to the execution order sequence of the node lock, and executes the corresponding shard task concurrently using multi-threading.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the asynchronous task processing method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the asynchronous task processing method as described in any one of claims 1 to 4.

Images