Data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data processing method and device, electronic equipment and a storage medium. The data processing method comprises the steps that in the execution process of subtasks, whether a preset Redis task lock exists or not is detected, the Redis task lock has expiration time, and the Redis task lock appears when execution of each subtask is completed; if yes, executing a timed task for querying the execution state of the parent task; if the parent task does not exist, the timed task is not executed, and the parent task comprises a plurality of subtasks. The method can utilize the Redis task lock to decouple the state updating frequency of the parent task and the subtask, avoid invalid query caused by overlarge execution completion time interval of the adjacent subtasks, and adopt the timed task to query the execution state of the parent task after detecting that the Redis task lock exists, thereby reducing the database performance pressure caused by frequent query of the parent task state, and improving the query efficiency of the database. Updating of the parent task execution state is triggered in time, and the updating efficiency of the parent task execution state is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, device, electronic device and storage medium. Background Art

[0002] A parent task usually has many subtasks, and the parent task needs to update its status to completed only when all subtasks are completed.

[0003] In the prior art, if the event trigger mode is used to update the status of the parent task, when m subtasks are completed in a very short time, each time a subtask is completed, the parent task query function needs to be triggered once to query the status of all subtasks. Limited by the query performance of the database (MySQL is generally used here), multiple parent task query functions are triggered in sequence or concurrently in a short period of time, which may bring greater performance pressure to the database. For example, when the time interval between the completion of two subtasks is less than 2s, performing two parent task queries may increase the data query pressure, and the query when the previous subtask is completed may be meaningless (invalid query). In order to reduce the frequency of parent task status query, each time the parent task query function is triggered, the completed tasks may be marked, and only the uncompleted subtasks are queried. However, if the completion time interval between the two subtasks is very large, the parent task status has not changed within this time period, and the parent task status query function triggered within this time period is meaningless, that is, there is an invalid query. If you use scheduled task polling to query whether there are any unfinished subtasks to update the parent task status, when the polling time is too long, and the completion of the last subtask happens to be triggered after the previous scheduled task is completed, it is necessary to wait for a long polling cycle before the next scheduled task can be executed. If the waiting time is too long, when all subtasks are completed, it may take some time to start the next scheduled task for parent task status query, and the update of the parent task status cannot be triggered in time, affecting data query efficiency; when the polling time is too short, a situation similar to the above event triggering mode will occur. Frequent queries on the execution status of subtasks to update the parent task status may bring greater performance pressure to the database. Summary of the invention

[0004] The present application aims to at least solve the technical problems in the prior art of database performance pressure caused by frequent queries on the parent task status, invalid queries caused by a long time interval between subtask executions, and failure to trigger timely updates of the parent task status due to long time intervals between scheduled tasks for parent task queries.

[0005] In order to solve the above technical problems, the present application provides a data processing method, including:

[0006] During the execution of the subtask, detect whether there is a preset Redis task lock, wherein the Redis task lock has an expiration time and the Redis task lock appears when each subtask is completed;

[0007] If it exists, execute the scheduled task to query the execution status of the parent task; if it does not exist, do not execute the scheduled task, wherein the parent task includes multiple child tasks.

[0008] In some embodiments, executing a scheduled task for querying the execution status of a parent task includes:

[0009] Regularly query the execution status of each subtask of the parent task;

[0010] Determining the execution result of the scheduled task according to the execution status of each of the subtasks;

[0011] The execution status of the parent task is updated according to the execution result of the scheduled task.

[0012] In some embodiments, updating the execution status of the parent task according to the execution result of the scheduled task includes:

[0013] If the execution result of the scheduled task is that all subtasks are completed, it is determined that the parent task is completed;

[0014] Update the execution status of the parent task to a completed status; or

[0015] If the execution result of the scheduled task is that all subtasks are not fully executed, it is determined that the parent task is not fully executed;

[0016] The execution status of the parent task is kept in an unfinished state.

[0017] In some embodiments, if the execution result of the scheduled task is that all subtasks are not fully executed, after determining that the parent task is not fully executed, the method further includes:

[0018] Detect whether the Redis task lock exists;

[0019] If it exists, execute the scheduled task again; if it does not exist, do not execute the scheduled task.

[0020] In some embodiments, the method further comprises:

[0021] When the execution of the initial subtask is triggered, the expiration time of the Redis task lock is set to N seconds;

[0022] After each subtask is executed, the expiration time of the Redis task lock is reset to the N seconds.

[0023] In some embodiments, the method further comprises:

[0024] Set the time interval between two consecutive scheduled task executions to M seconds, where N / 2<M<N.

[0025] In some embodiments, the method further comprises:

[0026] Cache the subtasks in redis to form redis cache data;

[0027] When each subtask is completed, the redis cache data corresponding to the subtask is marked;

[0028] The marking status of the subtask is queried periodically to determine the execution status of the parent task.

[0029] The present application also provides a data processing device, including:

[0030] A detection module is configured to detect whether there is a preset Redis task lock during the execution of the subtask, wherein the Redis task lock has an expiration time and the Redis task lock appears when each subtask is executed;

[0031] The execution module is configured to execute a scheduled task for querying the execution status of a parent task if the Redis task lock exists; and not execute the scheduled task if the Redis task lock does not exist, wherein the parent task includes multiple child tasks.

[0032] An embodiment of the present application also provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program on the memory.

[0033] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0034] The data processing method, device, electronic device and storage medium provided by the embodiments of the present application detect whether there is a preset Redis task lock during the execution of a subtask, and the Redis task lock has an expiration time, and the Redis task lock is reset when each subtask is executed; if the preset Redis task lock exists, it is determined that the execution status of the parent task may need to be updated, and a scheduled task for querying the execution status of the parent task is executed; if not, there is no need to execute the scheduled task, and the Redis task lock can be used to decouple the status update frequency of the parent and child tasks to avoid invalid queries caused by excessive time intervals between adjacent subtasks, and after detecting the existence of the Redis task lock, a scheduled task is used to query the execution status of the parent task, thereby reducing the database performance pressure caused by frequent queries of the parent task status, and timely triggering the update of the parent task execution status, thereby improving the update efficiency of the parent task execution status. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 A flowchart of a data processing method according to an embodiment of the present application;

[0037] Figure 2 Another flow chart of the data processing method according to an embodiment of the present application;

[0038] Figure 3 A schematic diagram of the structure of a data processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0040] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0041] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0042] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0043] It should also be understood that, although the present application has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0044] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0045] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0046] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0047] Figure 1 and Figure 2 FIG. 1 is a flow chart showing a data processing method provided by an embodiment of the present application. Figure 1 and Figure 2 As shown, the first embodiment of the present application provides a data processing method, including:

[0048] S101: During the execution of a subtask, detecting whether there is a preset Redis task lock, wherein the Redis task lock has an expiration time, and the Redis task lock appears when each subtask is executed.

[0049] A parent task includes multiple subtasks. The execution status of the parent task is determined according to the execution status of each subtask. When all subtasks of the parent task are completed, the execution status of the parent task is completed; when all subtasks of the parent task are not completed, for example, there are unexecuted subtasks, or some subtasks are still being executed, the execution status of the parent task is incomplete (the parent task is still being executed).

[0050] In this embodiment, Redis is used to pre-set a Redis task lock, which is named alive_key, and an expiration time of the Redis task lock is set. When the existence time of the Redis task lock exceeds the expiration time, the Redis task lock is automatically deleted or destroyed.

[0051] The Redis task lock appears when each subtask is completed, which means that the Redis task lock is reset when each task subtask is completed, so that it reappears with the above expiration time. That is, the Redis task lock is timed based on the previous completed subtask. For example, when the first subtask is completed, the Redis task lock appears, and disappears in a countdown manner after the time when the Redis task lock appears exceeds the above expiration time. When the time interval between the completion of the adjacent first subtask and the second subtask exceeds the above expiration time, the second subtask has not been completed, and the Redis task lock disappears automatically. After the second subtask is completed, the Redis task lock reappears. When the time interval between the completion of the adjacent first subtask and the second subtask does not exceed the above expiration time, when the second subtask after the first subtask is completed, the Redis task lock still does not disappear. After the second subtask is completed, the Redis task lock counts down again with the above expiration time, and disappears automatically after exceeding the above expiration time. In this step, during the execution of the subtask, it is detected whether there is a Redis task lock. If so, it is determined that the Redis task lock has not expired and is in a live state. The live state may be due to the fact that the previous subtask has been completed and the next subtask has not yet started, or the previous subtask has been completed and there are no unexecuted subtasks. If not, it is determined that the Redis task lock has expired and is in a non-living state. The non-living state may be due to a long time interval between the completion of the two subtasks, resulting in the completion of the previous subtask and the start of the next subtask. Or the time interval between the completion of the previous subtask and the start of the next subtask is long, the Redis task lock has expired, but the next subtask has not yet started.

[0052] As can be seen from the above, when the Redis task lock exists, it is determined that all subtasks may be executed to completion; and when the Redis task lock does not exist, it is determined that there are uncompleted tasks (including unexecuted and executing tasks) in the subtasks.

[0053] It is understandable that when multiple subtasks are executed at the same time, or the time interval between the execution of multiple subtasks is small (for example, less than 0.2s), the Redis task lock can be reset based on any one of the subtasks.

[0054] S102: If it exists, execute the scheduled task to query the execution status of the parent task; if it does not exist, do not execute the scheduled task.

[0055] By detecting whether the Redis task lock exists, it can be determined whether a scheduled task needs to be executed to query the execution status of the parent task and update the execution status of the parent task in a timely manner.

[0056] When it is determined that the Redis task lock exists, all subtasks may be executed and completed, and it is determined that the execution status of the parent task may need to be updated. Therefore, the current execution status of the parent task can be queried by executing a scheduled task, and then the execution status of the parent task can be updated. When it is determined that the Redis task lock does not exist, due to the existence of unfinished subtasks, it is determined that there is no need to update the execution status of the parent task. At this time, the execution status of the parent task remains in an unfinished state to reduce invalid queries. Among them, a scheduled task is a task that is executed according to time. This step is performed when it is determined that there is a Redis task lock. It can be determined that the time interval between the execution of two adjacent subtasks is short. At this time, a scheduled task query is used. There is no need to consider the completion of multiple subtasks at the same time or multiple subtasks are completed in a short period of time. Multiple concurrent queries or frequent queries are performed on the execution status of the parent task, which can effectively reduce the query pressure of the parent task execution status. At the same time, the execution time of the scheduled task can be set appropriately according to the expiration time of the Redis task lock to avoid frequent queries of the parent task status caused by too short polling time during scheduled queries, effectively reducing the performance pressure of the database, and using an ordinary database to meet the query requirements; the execution time of the above-mentioned scheduled task is appropriate, which can timely trigger the update of the parent task execution status and improve the update efficiency of the parent task execution status.

[0057] The data processing method provided by the embodiment of the present application detects whether there is a preset Redis task lock during the execution of a subtask, and the Redis task lock has an expiration time, and the Redis task lock appears when each subtask is executed; if the preset Redis task lock exists, it is determined that the execution status of the parent task may need to be updated, and a scheduled task for querying the execution status of the parent task is executed; if not, there is no need to execute the scheduled task, and the Redis task lock can be used to decouple the status update frequency of the parent and child tasks to avoid invalid queries caused by excessive time intervals between adjacent subtasks, and after detecting the existence of the Redis task lock, a scheduled task is used to query the execution status of the parent task, thereby reducing the database performance pressure caused by frequent queries of the parent task status, and timely triggering the update of the parent task execution status, thereby improving the update efficiency of the parent task execution status.

[0058] In some embodiments, in step S102, executing a scheduled task for querying the execution status of a parent task includes:

[0059] S1021: Periodically query the execution status of each subtask of the parent task;

[0060] S1022: Determine the execution result of the scheduled task according to the execution status of each of the subtasks;

[0061] S1023: Update the execution status of the parent task according to the execution result of the scheduled task.

[0062] Since the execution status of the parent task depends on the completion status of each subtask, in this embodiment, when executing a scheduled task, the execution status of each subtask can be checked and queried at regular intervals to determine whether all subtasks have been completed. For example, the execution status of each subtask is queried every 5 minutes, and the execution result of this scheduled task is determined based on the execution status of each subtask at each query.

[0063] The time interval of the scheduled task can be predetermined based on the type of each subtask and the task content. In a specific embodiment, when the subtask is a serial task, the average execution time of each serial task can be determined based on the number of tasks in the serial task and the execution time of each serial task, and then the time interval of the scheduled task can be determined based on the average execution time. For example, the time corresponding to twice the average execution time is determined as the time interval of the scheduled task. When the subtask is a parallel task, the time interval of the scheduled task can be determined based on the execution time of a task in the parallel task or based on the longest execution time of each parallel task. When the subtask contains both serial tasks and parallel tasks, the time interval of the scheduled task can be obtained by comprehensive consideration.

[0064] In another specific embodiment, when the subtask is a monitoring task, the monitoring task usually runs through the entire business operation. Therefore, the time interval of the scheduled task can be set to be longer, for example, querying the execution status of the subtask every 10 minutes; when the subtask is a sudden task, the time interval of the scheduled task can be set to be shorter, for example, querying the execution status of the subtask every 2 minutes.

[0065] The time interval of the above-mentioned scheduled task (execution time of the scheduled task) is the time interval for each polling of each subtask. The time interval of the scheduled task is determined according to the actual business conditions of the parent task and the subtask, and is not specifically limited in this application. In a specific implementation, the execution parameters such as the number of executions of the scheduled task, the execution rules (for example, whether all subtasks need to be polled), etc. can also be determined according to actual needs. After determining the execution result of the scheduled task based on the execution conditions of the above-mentioned subtasks, the execution status of the parent task is determined based on the execution result of the scheduled task, and then the execution status of the parent task is updated.

[0066] Specifically, Figure 2 As shown, in step S1023, the execution status of the parent task is updated according to the execution result of the scheduled task, including:

[0067] S201: If the execution result of the scheduled task is that all subtasks are completed, determine that the parent task is completed;

[0068] S202: updating the execution status of the parent task to a completed status; or

[0069] S203: If the execution result of the scheduled task is that all subtasks are not completely executed, determine that the parent task is not completely executed;

[0070] S204: Keep the execution status of the parent task as an unfinished state.

[0071] Scheduled tasks include two execution results, one is that all subtasks are fully executed, and the other is that all subtasks are not fully executed. If all subtasks are detected to be fully executed in this scheduled task, the parent task is determined to be completed, and the execution status of the parent task is updated to the completed state; if all subtasks are detected to be not fully executed in this scheduled task, the parent task is determined to be incomplete, and there is no need to update the execution status of the parent task, and the execution status of the parent task remains in the incomplete state.

[0072] It is understandable that during the execution of this scheduled task, the execution status of each subtask can be traversed and judged according to the execution order of each subtask. When it is detected that there are unfinished subtasks, it is determined that all subtasks are not fully executed.

[0073] In some embodiments, if the execution result of the scheduled task is that all subtasks are not fully executed, after determining that the parent task is not fully executed, the method further includes:

[0074] S301: Detect whether the Redis task lock exists;

[0075] S302: If it exists, execute the scheduled task again; if it does not exist, do not execute the scheduled task.

[0076] In this embodiment, the scheduled task is a task that can be executed in a scheduled loop multiple times. After the execution of the current scheduled task is completed, it can be determined based on the execution result of the current scheduled task that all subtasks have not been fully executed. At a certain interval, the next scheduled task is executed, and the above steps S1021 to S1023 are repeated. The execution status of each subtask of the parent task is periodically queried, and then according to the execution status of each subtask, it is determined whether the parent task has been completed. This cycle is repeated until the parent task is completed, the execution status of the parent task is updated to the completed status, and the loop is exited, ending the query and update of the execution status of the parent task.

[0077] like Figure 2 As shown, each time a scheduled task is executed, if the execution result of the scheduled task is that all subtasks are not fully executed, step S101 is repeated (step S301 is executed) to detect whether the Redis task lock exists, and whether to execute the scheduled task again is determined based on whether the Redis task lock exists, and the cycle is repeated until the parent task is completed. Before each scheduled task is executed, step S301 is used to detect whether the Redis task lock exists to ensure accurate judgment and improve the accuracy of updating the execution status of the parent task.

[0078] As can be seen from the above, in a timed task that is executed in multiple timed cycles, the time interval of the timed task can be the time interval for querying the execution status of each subtask each time when executing a single timed task (steps S1021 to S1023), or it can be the time interval between two adjacent executions of the timed task.

[0079] In a specific implementation, the scheduled task can be set according to actual needs. For example, when the number of subtasks is small and the relationship between the subtasks is simple, the scheduled task can be a single-executed scheduled task to query the execution status of the parent task, thereby improving the execution efficiency of the scheduled task. When the number of subtasks is large and the relationship between the subtasks is complex, the scheduled task can be set as a task that is executed in a scheduled loop multiple times (looping steps S301 to S302) to ensure the accuracy of the subtask status query and the accuracy of the parent task execution status query.

[0080] In this embodiment, by querying the execution status of the parent task through the timed task executed in the above-mentioned multiple timed cycles, a single timed task can be executed at each query, effectively avoiding simultaneous concurrent queries on the parent task status (concurrent access to the database during query) or frequent queries in a short period of time, effectively reducing data processing pressure and ensuring the data processing performance of the database.

[0081] In some embodiments, the method further comprises:

[0082] S401: When the execution of the initial subtask is triggered, the expiration time of the Redis task lock is set to N seconds;

[0083] S402: After each subtask is executed, the expiration time of the Redis task lock is reset to N seconds.

[0084] In this embodiment, the expiration time of the Redis task lock is set when the first subtask is triggered, for example, it is set to 5 seconds; each time a subtask is executed, the expiration time of the Redis task lock is reset to 5 seconds, that is, each time a subtask is executed, the expiration time of the Redis task lock is reset to make the judgment of the execution status update of the parent task more accurate. When it is determined that the Redis task lock exists, the execution of the scheduled task can be judged according to the execution status of the previous subtask, without the need to judge all the subtasks that have been executed. When it is determined that the Redis task lock does not exist, it can effectively avoid the invalid query (data invalidation processing) caused by the execution time interval between the two adjacent subtasks being too large, and still querying the execution status of the parent task.

[0085] When multiple subtasks are executed and completed at the same time, the expiration time of the Redis task lock can be reset to N seconds at the time when the execution is completed at the same time.

[0086] In some embodiments, the method further comprises:

[0087] Set the time interval between two consecutive scheduled task executions to M seconds, where N / 2<M<N.

[0088] For example, by setting the expiration time of the Redis task lock to 5 seconds and the time interval between two adjacent scheduled task executions to 3 seconds, after the last subtask is completed, the scheduled task that queries the execution status of the parent task can still be executed once, ensuring the accuracy of the execution status query of the parent task, and then accurately updating the execution status of the parent task based on the execution result of the last scheduled task.

[0089] In some embodiments, the method further comprises:

[0090] S501: Cache the subtask in redis to form redis cache data;

[0091] S502: When each subtask is completed, mark the redis cache data corresponding to the subtask;

[0092] S503: Periodically query the marking status of the subtask to determine the update status of the parent task.

[0093] In this embodiment, each subtask of the parent task can be cached in redis to form redis cache data. When each subtask is executed, the redis cache data corresponding to the subtask is marked. When the above-mentioned scheduled task is executed, it is only necessary to query the marking status of each subtask to quickly query the execution status of each subtask, thereby improving data query efficiency.

[0094] In particular, each time a scheduled task is executed, for the subtasks that have been marked in the previous scheduled task, since the subtasks have been completed, the next time the scheduled task is executed, there is no need to consider the subtasks that were marked last time. You only need to query the newly marked subtasks to determine whether all subtasks have been completed, further improving data query efficiency.

[0095] Figure 3 Schematic diagram of a data processing device according to an embodiment of the present application is shown. Figure 3 As shown, based on the above data processing method, the embodiment of the present application also provides a data processing device, including:

[0096] The detection module 100 is configured to detect whether there is a preset Redis task lock during the execution of the subtask, wherein the Redis task lock has an expiration time and the Redis task lock appears when each subtask is executed;

[0097] The execution module 200 is configured to execute a scheduled task for querying the execution status of a parent task if the Redis task lock exists; and not execute the scheduled task if the Redis task lock does not exist, wherein the parent task includes multiple child tasks.

[0098] In some embodiments, the execution module 200 includes:

[0099] A query unit, configured to periodically query the execution status of each subtask of the parent task;

[0100] A determination unit, configured to determine the execution result of the scheduled task according to the execution status of each of the subtasks;

[0101] The updating unit is configured to update the execution status of the parent task according to the execution result of the scheduled task.

[0102] In some embodiments, the updating unit is further configured to:

[0103] If the execution result of the scheduled task is that all subtasks are completed, it is determined that the parent task is completed;

[0104] Update the execution status of the parent task to a completed status; or

[0105] If the execution result of the scheduled task is that all subtasks are not fully executed, it is determined that the parent task is not fully executed;

[0106] The execution status of the parent task is kept in an unfinished state.

[0107] In some embodiments, after determining that the execution result of the scheduled task is that all subtasks are not fully executed and that the parent task is not fully executed, the updating unit

[0108] The detection module 100 is also configured to detect whether the Redis task lock exists;

[0109] The execution module 200 is also configured to execute the scheduled task again if the Redis task lock exists; if the Redis task lock does not exist, not execute the scheduled task.

[0110] In some embodiments, the data processing device further includes a setting module, including:

[0111] A setting unit, configured to set the expiration time of the Redis task lock to N seconds when the execution of the subtask is triggered;

[0112] The reset unit is configured to reset the expiration time of the Redis task lock to N seconds after detecting that any subtask is completed.

[0113] In some embodiments, the settings module is further configured to:

[0114] Set the time interval between two consecutive scheduled task executions to M seconds, where N / 2<M<N.

[0115] In some embodiments, the execution module 200 is further configured to:

[0116] Cache the subtasks in redis to form redis cache data;

[0117] When each subtask is completed, the redis cache data corresponding to the subtask is marked;

[0118] The marking status of the subtask is queried periodically to determine the execution status of the parent task.

[0119] Those skilled in the art will appreciate that the data processing device may include more or fewer components, for example, may also include an adjustment interface for a page display interface, a communication interface, etc., or may combine certain components, or arrange the components in different ways.

[0120] It should be noted that the data processing device provided in the embodiment of the present application corresponds to the data processing method in the above-mentioned embodiment. Based on the above-mentioned data processing method, technicians in this field can understand the specific implementation methods of the data processing device in the embodiment of the present application and its various variations. Any optional options in the data processing method embodiment are also applicable to the data processing device, which will not be repeated here.

[0121] An embodiment of the present application also provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program on the memory.

[0122] In some embodiments, the processor executing the computer program may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0123] The memory may be a read-only memory (ROM), a random access memory (RAM), a phase-change random access memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), an electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), a flash disk or other form of flash memory, a cache, a register, a static memory, a compact disk read-only memory (CD-ROM), a digital versatile disk (DVD) or other optical storage, a cassette or other magnetic storage device, or any other possible non-temporary medium used to store information or instructions that can be accessed by a computer device.

[0124] The electronic devices of the embodiments of the present application may include but are not limited to fixed terminal devices such as servers, desktop computers, digital TVs, etc., and mobile terminal devices such as vehicle-mounted devices (such as head-up display devices HUD), handheld devices (such as mobile phones, tablet computers, etc.), wearable devices (such as smart watches, smart bracelets, etc.), etc.

[0125] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0126] The computer-readable storage medium of the embodiment of the present application may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. In the embodiment of the present application, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device, for example, the above-mentioned memory.

[0127] The computer program of the embodiment of the present application can be organized into one or more computer executable components or modules. Any number and combination of such components or modules can be used to implement the various aspects of the present application. For example, the various aspects of the present application are not limited to the specific computer executable instructions or specific components or modules shown in the drawings and described herein. Other embodiments may include different computer executable instructions or components with more or less functions than those shown and described herein.

[0128] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A data processing method, It is characterized in that include: During the execution of the subtask, detect whether there is a preset Redis task lock, wherein the Redis task lock has an expiration time and the Redis task lock appears when each subtask is completed; If it exists, execute the scheduled task to query the execution status of the parent task; if it does not exist, do not execute the scheduled task, wherein the parent task includes multiple child tasks.

2. The method according to claim 1, It is characterized in that The execution of the scheduled task for querying the execution status of the parent task includes: Regularly query the execution status of each subtask of the parent task; Determining the execution result of the scheduled task according to the execution status of each of the subtasks; The execution status of the parent task is updated according to the execution result of the scheduled task.

3. The method according to claim 2, It is characterized in that The updating of the execution status of the parent task according to the execution result of the scheduled task includes: If the execution result of the scheduled task is that all subtasks are completed, it is determined that the parent task is completed; Update the execution status of the parent task to a completed status; or If the execution result of the scheduled task is that all subtasks are not fully executed, it is determined that the parent task is not fully executed; The execution status of the parent task is kept in an unfinished state.

4. The method according to claim 3, It is characterized in that If the execution result of the scheduled task is that all subtasks are not completely executed, after determining that the parent task is not completely executed, the method further includes: Check whether the Redis task lock exists; If it exists, execute the scheduled task again; if it does not exist, do not execute the scheduled task.

5. The method according to claim 1, It is characterized in that The method further comprises: When the execution of the initial subtask is triggered, the expiration time of the Redis task lock is set to N seconds; After each subtask is executed, the expiration time of the Redis task lock is reset to the N seconds.

6. The method according to claim 5, It is characterized in that The method further comprises: Set the time interval between two consecutive scheduled task executions to M seconds, where N / 2<M<N.

7. The method according to claim 1, It is characterized in that The method further comprises: Cache the subtasks in redis to form redis cache data; When each subtask is completed, the redis cache data corresponding to the subtask is marked; The marking status of the subtask is queried periodically to determine the execution status of the parent task.

8. A data processing device, It is characterized in that include: A detection module is configured to detect whether there is a preset Redis task lock during the execution of the subtask, wherein the Redis task lock has an expiration time and the Redis task lock appears when each subtask is executed; The execution module is configured to execute a scheduled task for querying the execution status of a parent task if the Redis task lock exists; and not execute the scheduled task if the Redis task lock does not exist, wherein the parent task includes multiple child tasks.

9. An electronic device, It is characterized in that The method comprises at least a memory and a processor, wherein the memory stores a computer program, and the processor performs the steps of the method according to any one of claims 1 to 7 when executing the computer program on the memory.

10. A computer-readable storage medium, It is characterized in that The computer-readable medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images