Data processing task distribution method and device, medium and electronic equipment

Abstract

Embodiments of the present application provide a data processing task distribution method and device, a medium and an electronic device. The method comprises: obtaining a task table comprising at least one task row, the task row being used to record a row-level lock state of the task row itself and a task state of a corresponding data processing task in real time, wherein the row-level lock state is in a locked state in the switching process of the task state; in response to an access request of an arbitrary task processing node, determining a task row in which the task state is an initial state and the row-level lock state is an unlocked state in the task table as a target task row; distributing the data processing task corresponding to the target task row to the arbitrary task processing node, and switching the task state of the target task row from the initial state to an execution state, so as to process the data processing task by the arbitrary task processing node. The present application can improve the orderliness of data processing task distribution.

Description

Technical Field

[0001] This application relates to the field of computer and data processing technology, and more specifically, to a data processing task distribution method, apparatus, medium, and electronic device. Background Technology

[0002] In the era of big data, single machines often face various bottlenecks in scenarios requiring simultaneous processing of massive amounts of data. Therefore, distributed tasks are increasingly being used. Breaking down a large task into several smaller tasks and distributing them across multiple machines is the most common solution. However, avoiding chaos in distributed task distribution remains a challenge. Therefore, improving the orderliness of data processing task distribution is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] The embodiments of this application provide a data processing task distribution method, apparatus, computer program product or computer program, computer medium and electronic device, which can at least to some extent improve the orderliness of data processing task distribution.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of the embodiments of this application, a data processing task distribution method is provided. The method includes: obtaining a pre-constructed task table, the task table including at least one task row, the task row being used to record in real time its own row-level lock state and the task state of the data processing task corresponding to the task row, wherein the row-level lock state is in a locked state during the switching of the task state; responding to an access request from any task processing node, determining a task row in the task table whose task state is in an initial state and whose row-level lock state is in an unlocked state as a target task row, the access request being used by the any task processing node to receive the data processing task; distributing the data processing task corresponding to the target task row to the any task processing node, and switching the task state of the target task row from the initial state to the execution state, so that the any task processing node can process the data processing task.

[0006] In some embodiments of this application, based on the foregoing scheme, before obtaining the pre-built task table, the method further includes: obtaining at least one data processing task; constructing a task table and creating task rows in the task table with the same number of tasks as the data processing task; writing the at least one data processing task into each task row of the task table; defining the row-level lock state of each task row in the task table as an unlocked state; and defining the task state of the data processing task corresponding to each task row as an initial state.

[0007] In some embodiments of this application, based on the foregoing scheme, the method further includes: defining the idempotency of the data processing tasks corresponding to each task row in the task table.

[0008] In some embodiments of this application, based on the foregoing scheme, determining a task row in the task table whose task state is in the initial state and whose row-level lock state is in the unlocked state as the target task row includes: polling the task rows in the task table, determining a task row whose task state is in the initial state as a candidate task row; if the row-level lock state of the candidate task row is in the locked state, then continuing to poll other task rows in the task table until a candidate task row whose row-level lock state is in the unlocked state is determined as the target task row.

[0009] In some embodiments of this application, based on the foregoing scheme, the method further includes: determining a task row in the task table whose task state is an initial state and whose row-level lock state is an unlocked state as a target task row, and then adjusting the row-level lock state of the target task row from an unlocked state to a locked state; after switching the task state of the target task row from the initial state to the execution state, adjusting the row-level lock state of the target task row from a locked state to an unlocked state.

[0010] In some embodiments of this application, based on the foregoing scheme, after switching the task status of the target task line from the initial state to the execution state, the method further includes: monitoring the processing result of any task processing node in processing the data processing task; if the processing result is success or failure, then switching the task status of the target task line from the execution state to the success state or failure state.

[0011] In some embodiments of this application, based on the foregoing scheme, after switching the task status of the target task line from the initial state to the execution state, the method further includes: monitoring the task status of the target task line; if the duration of the task status being in the execution state exceeds a preset time threshold, then switching the task status of the target task line from the execution state to the initial state.

[0012] According to one aspect of the embodiments of this application, a data processing task distribution apparatus is provided. The apparatus includes: an acquisition unit, configured to acquire a pre-built task table, the task table including at least one task row, the task row being configured to record in real time its own row-level lock state and the task state of the data processing task corresponding to the task row, wherein the row-level lock state is in a locked state during the switching of the task state; a determination unit, configured to, in response to an access request from any task processing node, determine a task row in the task table whose task state is in an initial state and whose row-level lock state is in an unlocked state, as a target task row, the access request being used by the any task processing node to receive the data processing task; and a distribution unit, configured to distribute the data processing task corresponding to the target task row to the any task processing node, and switch the task state of the target task row from the initial state to the execution state, so that the any task processing node can process the data processing task.

[0013] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer medium. A processor of a computer device reads the computer instructions from the computer medium and executes the computer instructions, causing the computer device to perform the methods described in the above embodiments.

[0014] According to one aspect of the embodiments of this application, a computer medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the above embodiments.

[0015] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to perform the method described in the above embodiments.

[0016] In this application, the row-level lock state and task status recorded in each task row of the task table reflect whether the corresponding data processing task has been distributed (i.e., whether it has been picked up by other task processing nodes) or is being executed. Therefore, using the row-level lock state of the task row and the task status of the corresponding data processing task as the basis for distributing data processing tasks to task processing nodes can prevent duplicate distribution of data processing tasks. Furthermore, any task processing node that picks up a data processing task will not pick up a task that has already been distributed or is being executed by another task processing node. In this way, the uniqueness of each data processing task distributed to a task processing node can be guaranteed, improving the orderliness of data processing task distribution and enhancing the stability of task data processing.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0019] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown;

[0020] Figure 2 A flowchart of a data processing task distribution method according to an embodiment of this application is shown;

[0021] Figure 3 An overall flowchart of data processing task distribution according to an embodiment of this application is shown;

[0022] Figure 4 A block diagram of a data processing task distribution apparatus according to an embodiment of the present application is shown;

[0023] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0024] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0025] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0026] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0027] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0028] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0029] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown.

[0030] like Figure 1 As shown, the system architecture may include task processing nodes (such as...) Figure 1The task processing servers 101, 102, and 103 shown (which could also be desktop computers, etc., but are not limited thereto, and this application does not impose any limitations on them) are network 104 and database server 105, used to store data processing tasks. Network 104 serves as the medium for providing a communication link between the task processing nodes and database server 105. Network 104 can include various connection types, such as wired communication links, wireless communication links, etc.

[0031] It should be noted that the data processing task distribution method provided in this application embodiment can be executed by the database server 105. Accordingly, the data processing task distribution device is generally set in the database server 105 to execute the data processing task distribution scheme provided in this application embodiment.

[0032] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0033] Figure 2 A flowchart of a data processing task distribution method according to an embodiment of this application is shown. This data processing task distribution method can be executed by a device with computing processing capabilities, such as a... Figure 1 The database server 105 shown is used for execution. (Refer to...) Figure 2 As shown, this data processing task distribution method includes at least steps 210 to 230, which are described in detail below:

[0034] In step 210, a pre-built task table is obtained. The task table includes at least one task row. The task row is used to record the row-level lock state of the task row itself and the task state of the data processing task corresponding to the task row in real time. The row-level lock state is locked during the switching of the task state.

[0035] In this application, a task table can be pre-built. Specifically, a database that supports row-level locking can be built first, such as MySQL-InnoDB, PostgreSQL, Oracle, etc., and then the task table can be built in the database.

[0036] It should be noted that row-level locking, also known as record locking, locks a specific row (or record) in a table, locking the row being operated on.

[0037] In this application, before obtaining the pre-built task table, the task table can be constructed by performing the following steps 201 to 204:

[0038] Step 201: Obtain at least one data processing task.

[0039] Step 202: Construct a task table and create task rows in the task table that are equal in number to the number of tasks in the data processing task.

[0040] Step 203: Write the at least one data processing task into the respective task rows of the task table.

[0041] Step 204: Define the row-level lock state of each task row in the task table as an unlocked state, and define the task state of the data processing task corresponding to each task row as the initial state.

[0042] To obtain at least one data processing task, for example, in a sales scenario, it is necessary to determine the predicted sales data for each product in each store over a certain period in the future. Based on this, stores can be grouped according to their characteristics, and the task of predicting the product sales data of each store can be broken down into multiple sub-tasks of product sales data prediction (i.e., multiple data processing tasks).

[0043] Furthermore, a task table is constructed, and task rows are created in the task table with the same number of tasks as the data processing task. For example, if the data processing task has 30 tasks, then 30 task rows are created in the task table. The task rows are used to record the row-level lock state of the task row itself and the task state of the corresponding data processing task in real time.

[0044] In this application, the task status of the data processing task can include four types, such as initial status, execution status, success status, and failure status.

[0045] Furthermore, after constructing the task table and creating task rows in the task table, the at least one data processing task can be written into each task row of the task table, and the row-level lock state of each task row in the task table is defined as an unlocked state, and the task state of the data processing task corresponding to each task row is defined as an initial state.

[0046] In this application, each task row can also be defined with a number as a task row number, which can be used to identify and distinguish the corresponding data processing task.

[0047] In this application, step 205 may also be performed:

[0048] Step 205: Define the idempotency of the data processing tasks corresponding to each task row in the task table.

[0049] It should be noted that idempotence is a mathematical and computer science concept. In programming, an idempotent operation ensures that the effect of a data processing task after any number of executions is the same as the effect of a single execution. In other words, in this application, even if a data processing task fails and needs to be reprocessed by the task processing node, the idempotency definition of the data processing tasks corresponding to each task row in the task table prevents the processing result from being influenced by previous processing results, thus ensuring the uniqueness and accuracy of the data processing result.

[0050] Continue to refer to Figure 2 In step 220, in response to an access request from any task processing node, a task row in the task table with the task status in the initial state and the row-level lock status in the unlocked state is determined as the target task row, and the access request is used by the any task processing node to retrieve the data processing task.

[0051] In this application, each data processing task in the database server needs to be distributed to each task processing node for processing. Based on this, each task processing node can send an access request to the database server to receive the data processing task.

[0052] Specifically, upon receiving an access request from any task processing node, a task row in the task table with the task status set to the initial state and the row-level lock status set to the unlocked state can be identified as the target task row. By identifying a task row in the task table with the task status set to the initial state and the row-level lock status set to the unlocked state, the uniqueness of each data processing task distributed to the task processing node during subsequent task distribution can be guaranteed.

[0053] Specifically, in this application, determining a task row in the task table whose task state is in the initial state and whose row-level lock state is in the unlocked state, as the target task row, can be performed according to the following steps 221 to 222:

[0054] Step 221: Poll the task rows in the task table and determine a task row whose task status is in the initial state as a candidate task row.

[0055] Step 222: If the row-level lock state of the candidate task line is locked, continue to poll other task lines in the task table until a candidate task line with an unlocked row-level lock state is determined as the target task line.

[0056] Continue to refer to Figure 2In step 230, the data processing task corresponding to the target task row is distributed to any one of the task processing nodes, and the task status of the target task row is switched from the initial state to the execution state so that any one of the task processing nodes can process the data processing task.

[0057] Specifically, during the polling of each task row in the task table, it is first determined whether the data processing task corresponding to the task row is in the initial state. If not, it indicates that the data processing task corresponding to the task row is being executed, has failed to be processed, or has been successfully processed. Then, the polling continues on other task rows in the task table. If yes, it indicates that the data processing task corresponding to the task row has not yet been executed by the task processing node. Then, the task row is taken as a candidate task row.

[0058] Furthermore, it is determined whether the row-level lock status corresponding to the candidate task row is locked. If it is, it indicates that the data processing task corresponding to the candidate task row has been taken by other task processing nodes, and the process continues to poll other task rows in the task table. If not, it indicates that the data processing task corresponding to the candidate task row has not yet been taken by other task processing nodes, and the candidate task row is taken as the target task row.

[0059] In this application, the row-level lock state and task status recorded in each task row of the task table reflect whether the corresponding data processing task has been distributed (i.e., whether it has been picked up by other task processing nodes) or is being executed. Therefore, using the row-level lock state of the task row and the task status of the corresponding data processing task as the basis for distributing data processing tasks to task processing nodes can prevent duplicate distribution of data processing tasks. Furthermore, any task processing node that picks up a data processing task will not pick up a task that has already been distributed or is being executed by another task processing node. In this way, the uniqueness of each data processing task distributed to a task processing node can be guaranteed, improving the orderliness of data processing task distribution and enhancing the stability of task data processing.

[0060] In this application, during the polling of task rows in the task table, task rows can be polled randomly. Alternatively, task rows in the task table can be polled in ascending or descending order of their numbers. This approach improves the orderliness of data processing task distribution to some extent, facilitates the management of data processing task distribution, and ultimately enhances the efficiency of data processing task distribution.

[0061] Furthermore, in this application, steps 241 to 242 may also be performed:

[0062] Step 241: After determining a task row in the task table whose task state is the initial state and whose row-level lock state is the unlocked state as the target task row, adjust the row-level lock state of the target task row from the unlocked state to the locked state.

[0063] Step 242: After switching the task status of the target task line from the initial state to the execution state, adjust the row-level lock status of the target task line from the locked state to the unlocked state.

[0064] In this application, after determining the target task row in the task table, the row-level lock state of the target task row is changed from an unlocked state to a locked state. This ensures that when other task processing nodes poll the target task row, because the row-level lock state is locked, they will not be assigned data processing tasks to that target task row, thus preventing data processing tasks from being distributed to multiple task processing nodes. Furthermore, after switching the task state of the target task row from the initial state to the execution state, the row-level lock state of the target task row is changed from locked to unlocked. At this point, even if other task processing nodes poll the target task row, because the task state of the corresponding data processing task is in the execution state rather than the initial state, they will not be assigned data processing tasks to that target task row, similarly preventing data processing tasks from being distributed to multiple task processing nodes.

[0065] In summary, this application, by connecting the timing of the row-level lock state switching of a task row with the timing of the task state switching of the data processing task corresponding to the task row, ensures that during the time when a task row is determined as the target task row and the data processing task corresponding to that task row is being executed, multiple task processing nodes will not simultaneously receive the data processing task corresponding to that task row. This guarantees the uniqueness of each data processing task being distributed to the task processing node, thereby improving the orderliness of data processing task distribution and enhancing the stability of task data processing.

[0066] In this application, after switching the task status of the target task line from the initial state to the execution state, the following steps 251 to 252 may also be performed:

[0067] Step 251: Monitor the processing result of any one of the task processing nodes in processing the data processing task.

[0068] Step 252: If the processing result is success or failure, then the task status of the target task line is switched from execution status to success status or failure status.

[0069] In this application, by monitoring the processing of the data processing task by any one of the task processing nodes, and switching the task status of the target task line from the execution state to the success state or failure state when the processing result is success or failure, the computing resources of any one of the task processing nodes can be released in a timely manner, so that any one of the task processing nodes will not be occupied for a long time, and can participate in the processing of other data processing tasks in a timely manner, thereby improving the processing efficiency of data processing tasks.

[0070] In this application, after switching the task status of the target task line from the initial state to the execution state, the following steps 261 to 262 can also be performed:

[0071] Step 261: Monitor the task status of the target task line.

[0072] Step 262: If the duration of the task status being in the execution state exceeds a preset time threshold, then the task status of the target task line is switched from the execution state to the initial state.

[0073] In this application, by monitoring the task status of the target task line, and switching the task status of the target task line from the executing state to the initial state when the duration of the task status in the executing state exceeds a preset time threshold, the situation where the data processing task corresponding to the target task line cannot be successfully processed for a long time due to insufficient computing power of the task processing node or other reasons can be avoided. This allows other task processing nodes to take over the data processing task corresponding to the target task line in a timely manner until the data processing task is successfully processed, thereby improving the efficiency of the data processing task.

[0074] To enable those skilled in the art to better understand this application, the following is combined with Figure 3 The following is an illustration using a specific example.

[0075] See Figure 3 This illustrates an overall flowchart of data processing task distribution according to an embodiment of the present application.

[0076] Depend on Figure 3 Understandably, this application utilizes the row-level locking mechanism of the database to ensure that a data processing task can only be received by one task processing node at a time. This ensures the uniqueness of each data processing task being distributed to and processed by the task processing node, thereby improving the orderliness of data processing task distribution and enhancing the stability of task data processing.

[0077] In addition, since each task processing node can access the database to retrieve data processing tasks for processing, the scalability and flexibility of data processing tasks are greatly increased.

[0078] The following describes an apparatus embodiment of this application, which can be used to execute the data processing task distribution method described above in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the data processing task distribution method described above in this application.

[0079] Figure 4 A block diagram of a data processing task distribution apparatus according to an embodiment of this application is shown.

[0080] Reference Figure 4 As shown, a data processing task distribution apparatus 400 according to an embodiment of this application includes: an acquisition unit 401, a determination unit 402, and a distribution unit 403.

[0081] The acquisition unit 401 is used to acquire a pre-built task table, which includes at least one task row. The task row is used to record its own row-level lock state and the task state of the data processing task corresponding to the task row in real time. The row-level lock state is locked during the switching of the task state. The determination unit 402 is used to determine a task row in the task table whose task state is in the initial state and whose row-level lock state is unlocked, in response to an access request from any task processing node, as the target task row. The access request is used by the any task processing node to receive the data processing task. The distribution unit 403 is used to distribute the data processing task corresponding to the target task row to the any task processing node and switch the task state of the target task row from the initial state to the execution state so that the any task processing node can process the data processing task.

[0082] In some embodiments of this application, based on the foregoing scheme, the apparatus further includes: a construction unit, configured to acquire at least one data processing task before acquiring a pre-constructed task table; construct a task table and create task rows in the task table with the same number of tasks as the data processing tasks; write the at least one data processing task into each task row of the task table; define the row-level lock state of each task row in the task table as an unlocked state; and define the task state of the data processing task corresponding to each task row as an initial state.

[0083] In some embodiments of this application, based on the foregoing scheme, the construction unit is configured to: define the data processing tasks corresponding to each task row in the task table as idempotent.

[0084] In some embodiments of this application, based on the foregoing scheme, the determining unit 402 is configured to: poll the task rows in the task table, determine a task row whose task state is in the initial state, and take it as a candidate task row; if the row-level lock state of the candidate task row is in the locked state, then continue to poll other task rows in the task table until a candidate task row whose row-level lock state is in the unlocked state is determined as the target task row.

[0085] In some embodiments of this application, based on the foregoing scheme, the apparatus further includes: a switching unit, configured to determine a task row in the task table whose task state is an initial state and whose row-level lock state is an unlocked state, as a target task row, and then adjust the row-level lock state of the target task row from an unlocked state to a locked state; after switching the task state of the target task row from the initial state to the execution state, adjust the row-level lock state of the target task row from a locked state to an unlocked state.

[0086] In some embodiments of this application, based on the foregoing scheme, the switching unit is configured to: after switching the task status of the target task line from the initial state to the execution state, monitor the processing result of any task processing node in processing the data processing task; if the processing result is success or failure, switch the task status of the target task line from the execution state to the success state or failure state.

[0087] In some embodiments of this application, based on the foregoing scheme, the switching unit is configured to: monitor the task status of the target task line after switching the task status of the target task line from the initial state to the execution state; if the duration of the task status being in the execution state exceeds a preset time threshold, then switch the task status of the target task line from the execution state back to the initial state.

[0088] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0089] It should be noted that, Figure 5 The computer system 500 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0090] like Figure 5As shown, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 502 or programs loaded from storage portion 508 into Random Access Memory (RAM) 503, such as performing the methods described in the above embodiments. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0091] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a local area network (LAN) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0092] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs various functions defined in the system of this application.

[0093] It should be noted that the computer medium shown in the embodiments of this application can be a computer-readable signal medium, a computer medium, or any combination of the two. The computer medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer medium other than a computer medium, which can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0095] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0096] In another aspect, this application also provides a computer program product or computer program that includes computer instructions stored in a computer medium. A processor of a computer device reads the computer instructions from the computer medium and executes the computer instructions, causing the computer device to perform the methods described in the above embodiments.

[0097] In another aspect, this application also provides a computer medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0098] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0099] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0100] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0101] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data processing task distribution method, characterized in that, The method includes: Obtain a pre-built task table, which includes at least one task row. The task row is used to record the row-level lock state of the task row itself and the task state of the data processing task corresponding to the task row in real time. The row-level lock state is locked during the switching of the task state. In response to an access request from any task processing node, a task row in the task table with the task status in the initial state and the row-level lock status in the unlocked state is determined as the target task row. The access request is used by any task processing node to retrieve the data processing task. The data processing task corresponding to the target task row is distributed to any one of the task processing nodes, and the task status of the target task row is switched from the initial state to the execution state, so that any one of the task processing nodes can process the data processing task. After determining a task row in the task table that has an initial state and an unlocked row-level lock state as the target task row, the row-level lock state of the target task row is changed from unlocked to locked. After switching the task state of the target task row from the initial state to the execution state, the row-level lock state of the target task row is changed from locked to unlocked.

2. The method according to claim 1, characterized in that, Before obtaining the pre-built task table, the method further includes: Obtain at least one data processing task; Construct a task table, and create task rows in the task table that are equal in number to the number of tasks in the data processing task; Write the at least one data processing task into the respective task row of the task table; Define the row-level lock state of each task row in the task table as an unlocked state, and define the task state of the data processing task corresponding to each task row as the initial state.

3. The method according to claim 2, characterized in that, The method further includes: Idempotency is defined for the data processing tasks corresponding to each task row in the task table.

4. The method according to claim 1, characterized in that, The step of determining a task row in the task table whose task state is the initial state and whose row-level lock state is unlocked, as the target task row, includes: Poll the task rows in the task table to determine a task row whose task status is in the initial state, and use it as a candidate task row; If the row-level lock of the candidate task line is in a locked state, then continue to poll other task lines in the task table until a candidate task line in an unlocked state is determined as the target task line.

5. The method according to claim 1, characterized in that, After switching the task status of the target task line from the initial state to the execution state, the method further includes: Monitor the processing result of the data processing task by any of the task processing nodes; If the processing result is success or failure, the task status of the target task line will be switched from execution status to success status or failure status.

6. The method according to claim 5, characterized in that, After switching the task status of the target task line from the initial state to the execution state, the method further includes: Monitor the task status of the target task line; If the duration of the task being in the execution state exceeds a preset time threshold, the task state of the target task line will be switched from the execution state to the initial state.

7. A data processing task distribution device, characterized in that, The device includes: The acquisition unit is used to acquire a pre-built task table, the task table including at least one task row, the task row being used to record the row-level lock state of the task row itself in real time, and the task state of the data processing task corresponding to the task row, wherein the row-level lock state is in a locked state during the switching of the task state. The determining unit is used to respond to an access request from any task processing node by determining a task row in the task table whose task status is in the initial state and whose row-level lock status is in the unlocked state, as the target task row. The access request is used by the any task processing node to retrieve the data processing task. The distribution unit is used to distribute the data processing task corresponding to the target task row to any one of the task processing nodes, and switch the task status of the target task row from the initial state to the execution state, so that any one of the task processing nodes can process the data processing task. After determining a task row in the task table that has an initial state and an unlocked row-level lock state as the target task row, the row-level lock state of the target task row is changed from unlocked to locked. After switching the task state of the target task row from the initial state to the execution state, the row-level lock state of the target task row is changed from locked to unlocked.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to perform the operations performed by the method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, The electronic device includes one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to perform the operations performed by the method as described in any one of claims 1 to 6.

Images