Data state-based timing task data processing method, device and medium

Abstract

Embodiments of the present application provide a data state-based timing task data processing method and device, equipment and a storage medium, relating to the technical field of data processing. The method comprises: obtaining first state data from a database according to a first table through a first process; transferring the obtained first state data from the first table to a second table through the first process; sending the second state data to a second process through the first process; processing the second state data based on a preset task through the second process; and transferring the processed second state data from the second table to a third table through the first process, so that the second state data is converted into third state data. The embodiments of the present application avoid the disadvantages caused by using a flag field to identify the processing state of data in a data table, and reduce the time consumption when searching for unprocessed data in the data processing process.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, device and storage medium for timed task data processing based on data status. Background Technology

[0002] Automation is a widely used data processing method. Enterprises generate a large amount of data every day. If this data needs to be filtered, integrated, and calculated within the same day, technicians will usually develop scheduled tasks to complete these operations.

[0003] Processing daily data in batches is a common type of automation task. Typically, this data is stored in a single database table. If the processed data only involves adding or updating a few fields, this table is highly likely to be reused to store the processed data. Thus, unprocessed and processed data occupy the same table. Developers usually introduce a flag field to indicate the status of a record (unprocessed, in transit, processed, etc.). This design has a drawback: at the start of each scheduled task, based on the batch processing setting, the task needs to scan the database table for a certain number of records with the status "unprocessed." Considering the extremely low distinguishability of the flag field and the inefficiency of adding an index, scanning this unprocessed data incurs a time penalty. Furthermore, the larger the total amount of data stored in the database table, the longer the search time will be. Summary of the Invention

[0004] The main objective of this application is to propose a method, apparatus, device, and storage medium for timed task data processing based on data status, aiming to solve the technical problem of excessive time consumption caused by acquiring unprocessed state data when timed task data processing.

[0005] In a first aspect, embodiments of this application propose a method for processing timed task data based on data status, the method comprising the following steps:

[0006] In response to the start of a preset timed data processing task, the first process retrieves first state data from the database according to a first table, wherein the first table is used to record the first state data;

[0007] The first process transfers the acquired first state data from the first table to the second table, thereby converting the first state data into second state data. The second table is used to record the second state data.

[0008] The second status data is sent to the second process through the first process;

[0009] The second process processes the second state data based on a preset task and returns the processed second state data to the first process.

[0010] The first process transfers the processed second state data from the second table to the third table, thereby converting the second state data into third state data. The third table is used to record the third state data.

[0011] In conjunction with the first aspect, in one possible implementation of the first aspect, after retrieving the first state data from the first table of the database via the first process, the method further includes:

[0012] When the first process fails to acquire the first state data, the failed acquisition of the first state data is transferred to the fourth table, so that the first state data is converted into the fourth state data, wherein the fourth table is used to record the fourth state data.

[0013] In conjunction with the first aspect, in one possible implementation of the first aspect, after the second process processes the second state data based on a preset task, the method further includes:

[0014] When the second process fails to process the second state data, the failed second state data is transferred to the fourth table, so that the second state data is converted into fourth state data, wherein the fourth table is used to record the fourth state data.

[0015] In conjunction with the first aspect, in one possible implementation of the first aspect, after sending the second state data to the second process via the first process, the method further includes:

[0016] The first process receives a confirmation message returned by the second process within a preset time interval, the confirmation message indicating that the second process has received the second status data;

[0017] If no confirmation message is received within the preset time interval, the first process transfers the second status data to the fourth table, thereby converting the second status data into fourth status data, wherein the fourth table is used to record the fourth status data.

[0018] In conjunction with the first aspect, in one possible implementation of the first aspect, after the first process receives an acknowledgment message returned by the second process within a preset time interval, the method further includes:

[0019] The second status data that has been received by the second process is determined based on the confirmation message;

[0020] The first process releases the second state data that has been received by the second process.

[0021] In conjunction with the first aspect, in one possible implementation of the first aspect, the step of transferring the acquired first state data from the first table to the second table through the first process includes:

[0022] The first state data obtained through the first process is deleted from the first table;

[0023] The first status data that has been deleted from the first table is added to the second table.

[0024] In conjunction with the first aspect, in one possible implementation of the first aspect, the step of transferring the processed second state data from the second table to the third table through the first process includes:

[0025] The processed second status data is deleted from the second table through the first process;

[0026] The second status data that will be deleted from the second table will be added to the third table.

[0027] Secondly, embodiments of this application propose a timed task data processing apparatus based on data status, the apparatus comprising:

[0028] The data acquisition module is used to respond to the start of a preset timed data processing task, and to obtain first state data from the database through a first process according to a first table, wherein the first table is used to record the first state data;

[0029] The first data transfer module is used to transfer the first state data obtained by the first process from the first table to the second table, so that the first state data is converted into second state data, wherein the second table is used to record the second state data;

[0030] A sending module is used to send the second status data to the second process through the first process;

[0031] The processing module is used to process the second state data through the second process based on a preset task, and return the processed second state data to the first process;

[0032] The second data transfer module is used to transfer the processed second state data from the second table to the third table through the first process, so that the second state data is converted into third state data, wherein the third table is used to record the third state data.

[0033] Thirdly, embodiments of this application propose an electronic device, including a memory and a processor, wherein the memory stores a computer program or instructions, and the processor executes the computer program or instructions to implement the timed task data processing method based on data status as described in the first aspect above.

[0034] Fourthly, embodiments of this application propose a computer-readable storage medium storing a computer program or instructions, which, when executed by a processor, implements the timed task data processing method based on data state as described in the first aspect above.

[0035] The data processing method, apparatus, device, and storage medium based on data status proposed in this application first respond to the start of a preset timed data processing task. A first process retrieves first status data from a database according to a first table, where the first table records the first status data. This concentrates unprocessed data in the first table, completely eliminating the risk of task timeout due to excessive time consumption as the total amount of data in the data table increases. Then, the first process transfers the retrieved first status data from the first table to a second table, converting the first status data into second status data, where the second table records the second status data. The first process sends the second status data to a second process, thus concentrating in-process data in the second table. The second process processes the second status data based on a preset task and returns the processed second status data to the first process. The first process then transfers the processed second status data from the second table to a third table, converting the second status data into third status data, where the third table records the third status data. The solution in this application embodiment records data in different states through different database tables, avoiding the drawbacks of using flag fields to identify the processing status of data in the data tables, and reducing the time consumption generated when searching for unprocessed data in the data processing flow. Attached Figure Description

[0036] Figure 1a This is a diagram illustrating the relationship between average data processing time and daily data volume in scheduled data processing tasks.

[0037] Figure 1b This is a schematic diagram illustrating the data relationship between data processing time and data offset in a scheduled data processing task.

[0038] Figure 2 This is a flowchart illustrating the timed task data processing method based on data status provided in an embodiment of this application.

[0039] Figure 3 This is a schematic diagram illustrating the process of the first process acquiring first state data provided in an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of data state transition during the timed task data processing process provided in the embodiments of this application;

[0041] Figure 5 This is a schematic diagram of the structure of the timed task data processing device based on data status provided in the embodiments of this application;

[0042] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0046] Automation is a widely used data processing method. Enterprises generate a large amount of data every day. If this data needs to be filtered, integrated, and calculated within the same day, technicians will usually develop scheduled tasks to complete these operations.

[0047] Processing daily data in batches is a common type of automation task. Typically, this data is stored in a single database table. If the processed data only involves adding or updating a few fields, this table is highly likely to be reused to store the processed data. Thus, unprocessed and processed data occupy the same table. Developers usually introduce a flag field to indicate the status of a record (unprocessed, in transit, processed, etc.). This design has a drawback: at the start of each scheduled task, based on the batch processing setting, the task needs to scan the database table for a certain number of records with the status "unprocessed." Considering the extremely low distinguishability of the flag field and the inefficiency of adding an index, scanning this unprocessed data incurs a time penalty. Furthermore, the larger the total amount of data stored in the database table, the longer the search time will be.

[0048] For example, a scheduled task in an intelligent outbound call robot system needs to process a batch of data. Data meeting specific conditions is retrieved from the database (DB) by process 1 (P1) and then passed to process 2 (P2) for processing. The processed data is then written back to process 1 by process 2 and stored in the database. When process 1 queries the database, the problem described above arises: unprocessed and processed data share the same table. Therefore, this table defines a flag field to identify the status of a record (unprocessed, in transit, processed, etc.). Each time the scheduled task runs, process 1's operation of retrieving a batch of unprocessed data from the table has a time complexity of O(n). As time progresses, new data is added to this table daily, and the position of the data needed by process 1 will continuously shift, requiring more and more data to be scanned in the database, resulting in increasingly higher time overhead. Figure 1a The diagram showing the relationship between average data processing time and daily data volume is as follows: Figure 1b The diagram illustrating the data processing time versus data offset shows that when the scheduled task first retrieves data from the table, it only takes 4 seconds for every 300 records. However, when retrieving the 60,000th record of the day, the processing time increases to 16 seconds. As the daily data volume increases, the average time consumption of this scheduled task also increases; the time consumed for the nth execution of the scheduled task each day also increases with increasing n. The processing time exhibits a time complexity of O(n) as the data offset increases. Although the current execution time of the scheduled task is still within an acceptable range, when the total data volume reaches a certain level, the processing time will exceed the scheduled task's execution interval, causing the scheduled task to be blocked and delayed, preventing the completion of the daily data processing task, making it an unacceptable defect. The task's processing time should not increase linearly with the increase in data volume; therefore, the time complexity of the data retrieval operation must be optimized to below O(n).

[0049] In view of this, embodiments of this application provide a method, apparatus, device, and storage medium for timed task data processing based on data status, aiming to solve the technical problem of excessive time consumption caused by acquiring unprocessed state data when timed tasks process data.

[0050] The timed task data processing method, apparatus, device, and storage medium based on data status provided in this application are specifically described through the following embodiments. First, the timed task data processing method based on data status in this application embodiment is described.

[0051] The data processing method for timed tasks based on data status provided in this application relates to the field of data processing technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the data processing method for timed tasks based on data status, but is not limited to the above forms.

[0052] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0053] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards of the relevant countries and regions. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data for the proper functioning of the embodiments of this application obtained.

[0054] Please see Figure 2 , Figure 2 This is a flowchart illustrating a timed task data processing method based on data status, provided as an embodiment of this application. Figure 2 As shown in the figure, the present application provides a timed task data processing method based on data status, including but not limited to steps S110-S150, which will be described in turn below.

[0055] Step S110: In response to the start of a preset timed data processing task, the first process retrieves first state data from the database according to the first table, wherein the first table is used to record the first state data.

[0056] It is understood that a scheduled data processing task is a data processing task that is executed periodically based on a pre-set time interval. For example, a data processing task may be executed once an hour, once a day, or once a week. This application does not specifically limit the execution time interval of the scheduled data processing task.

[0057] It is understood that the first state data described in the embodiments of this application represents data in an unprocessed state. The process of generating the first state data can be as follows: the start time of the previous round of data processing task is recorded as T1, and the start time of the current round of data processing task is recorded as T2. Then, the data to be processed generated by the business system between T1 and T2 can be used as the first state data.

[0058] It is understood that the first table in this application embodiment is a database table used to record data in the database that is in an unprocessed state (i.e., first state data). Specifically, there are multiple first state data in the first table, and the first table records information such as the data identifier, storage location, storage time, and data size of each first state data. Furthermore, the first state data can be sorted sequentially according to storage time.

[0059] For example, the data to be processed generated by the business system between the start time T1 of the previous data processing task and the start time T2 of the current data processing task includes data Data1. Subsequently, data Data1 is stored in the database, and information such as data identifier, storage location, storage time, and data size of data Data1 is recorded in the first table.

[0060] It is understood that when the start time of the timed data processing task is reached according to the preset time interval, the timed data processing task is started. In this embodiment of the application, the unprocessed first state data is obtained through the first process.

[0061] Please see Figure 3 , Figure 3 This diagram illustrates the process by which the first process acquires the first state data. Figure 3 As shown, the process by which the first process obtains the first state data is as follows:

[0062] Step S201: The first process obtains the first table;

[0063] Step S202: The first process determines a preset first quantity of first state data from the first table according to the storage time from earliest to latest;

[0064] Step S203: For the first state data to be obtained, the first process determines the storage location corresponding to the first state data in the first table, and obtains the corresponding first state data from the database according to the storage location.

[0065] Here, the first quantity represents the data capacity that this round of data processing task can process. For example, if each round of data processing task is configured to process a maximum of 100,000 data points, then when executing this round of data processing task, the first process will obtain a maximum of 100,000 first state data points from the first table, and based on the storage time recorded in the first table, the first state data with the earlier storage time will be obtained first.

[0066] In practice, a counter can be set up. The counter is cleared to zero at the beginning of each round of data processing tasks. The counter is incremented by one for each first state data item retrieved from the database by the first process. When the counter reaches a first count, the first process stops retrieving first state data from the database.

[0067] In this embodiment of the application, unprocessed first state data is recorded in a first table. This eliminates the need to use a flag field in the data table to identify the processing status of the data and also omits the original process of scanning unprocessed data based on the flag field, thus simplifying the time complexity of querying unprocessed data.

[0068] Step S120: The first state data obtained is transferred from the first table to the second table through the first process, so that the first state data is converted into second state data, wherein the second table is used to record the second state data.

[0069] It is understandable that the first state data that has been transmitted to the first process can be classified as data in transit. That is, the data has been converted from an unprocessed state to a transit state. Therefore, after the first process obtains the first state data from the database, it transfers the obtained first state data from the first table to the second table. The second table is used to record the second state data in the database. The second state data represents the data in transit. "In transit state" can be understood as data that is in a process.

[0070] For example, transferring the acquired first state data from the first table to the second table through the first process includes the following steps: deleting the acquired first state data from the first table through the first process, and adding the first state data deleted from the first table to the second table.

[0071] Understandably, after the first process retrieves the first state data from the database, it first deletes the first state data from the first table, and then adds the first state data to the second table, thus completing the data state transition.

[0072] Step S130: Send the second status data to the second process through the first process.

[0073] It is understood that in the embodiments of this application, the first process obtains unprocessed data from the database, and after obtaining the unprocessed data, sends the data to the second process for processing.

[0074] Step S140: The second process processes the second state data based on a preset task and returns the processed second state data to the first process.

[0075] Understandably, the second process executes a preset task to process the second state data received from the first process. This preset task type can be operations such as data verification, calculation, or business processing; this embodiment does not specifically limit the preset task type. After processing the second state data, the second process sends the processed second state data back to the first process.

[0076] Step S150: The processed second state data is transferred from the second table to the third table through the first process, so that the second state data is converted into third state data, wherein the third table is used to record the third state data.

[0077] Understandably, the first process receives the processed second-state data from the second process. At this point, the data can be categorized as processed, and then the processed data is stored in the database. Specifically, the first process transfers the processed second-state data from the second table to the third table, completing the switch from the in-transit state to the processed state. The third table is used to record the data in the database that is in the processed state. The third-state data described in this embodiment represents the processed data.

[0078] For example, the process of transferring the processed second state data from the second table to the third table through the first process includes the following steps: deleting the processed second state data from the second table through the first process, and adding the second state data deleted from the second table to the third table.

[0079] For example, the first process sends the second state data, data1, to the second process, which processes data1 according to a preset task. After processing data1, the second process sends the processed data1 back to the first process, which then deletes data1 from the second table and adds it to the third table. In this way, data1 changes from its in-transit state to its processed state.

[0080] In some possible implementations, after obtaining first state data from a first table in a database through a first process, the method further includes: when the first process fails to obtain the first state data, transferring the failed first state data to a fourth table, thereby converting the first state data into fourth state data, wherein the fourth table is used to record the fourth state data.

[0081] For example, transferring the first status data of the failed acquisition to the fourth table includes the following steps: deleting it from the first table and adding the first status data that will be deleted from the first table to the fourth table.

[0082] Understandably, if the first process fails to retrieve certain first-state data from the database, then this first-state data can be classified as error data. That is, the data should be changed from an unprocessed state to an error state, and then the data in the error state should be deleted from the first table and added to the fourth table. In this embodiment, the fourth table is used to record data in the database that is in an error state; the fourth-state data represents data in an error state.

[0083] For example, if the first table records the storage location of data1, then the first process searches for data1 in the database based on that storage location. If data1 is not found, it can be confirmed that the retrieval of data1 has failed. Then, data1 is deleted from the first table and added to the fourth table, so that data1 changes from its original unprocessed state to an error state.

[0084] In some possible embodiments, after the second state data is processed by a second process based on a preset task, the method of this application embodiment may further include:

[0085] When the second process fails to process the second state data, the failed second state data is transferred from the second table to the fourth table, so that the second state data is transformed into the fourth state data.

[0086] For example, transferring the second status data that failed to be processed from the second table to the fourth table specifically includes the following steps: deleting the second status data that failed to be processed from the second table and adding the deleted second status data to the fourth table.

[0087] Understandably, if the second process fails to process the second state data, the failed second state data can be classified as error data. That is, the data should be changed from the in-transit state to the error state, and then the data in the error state should be deleted from the second table and added to the fourth table.

[0088] In some possible embodiments, after the second state data is sent to the second process by the first process, the method further includes: the first process receiving an acknowledgment message returned by the second process within a preset time interval, the acknowledgment message indicating that the second process has received the second state data.

[0089] Understandably, after receiving the second status data from the first process, the second process returns an acknowledgment message to the first process to notify it that it has received the second status data. The acknowledgment message returned by the second process to the first process may include information such as the status identifier of the second status data, so that the first process can recognize that the second status data has been received by the second process.

[0090] Understandably, after the first process sends the second status data to the second process, it starts timing with a timer. If the timing exceeds the preset time interval and no confirmation message is received from the second process, it is assumed that the second process has not received the second status data.

[0091] In some possible embodiments, if the first process does not receive an acknowledgment message for the second state data within a preset time interval, the first process transfers the second state data to the fourth table, thereby converting the second state data into the fourth state data.

[0092] For example, the first process transfers the second state data to the fourth table by the following steps: the first process deletes the second state data from the second table and adds the deleted second state data to the fourth table.

[0093] For example, the first process sends the second state data data1 to the second process and starts a counter, setting a preset time interval of T3. If the first process receives an acknowledgment message from the second process for data1 within the time interval T3, it is determined that the second process has successfully received data1; otherwise, it is determined that the second process has not received data1, and then data1 is deleted from the second table and added to the fourth table, so that data1 changes from the original in-transit state to the error state.

[0094] In some possible embodiments, after the first process receives an acknowledgment message returned by the second process within a preset time interval, the method of this application embodiment further includes: determining the second state data that has been received by the second process based on the acknowledgment message, and the first process releasing the second state data that has been received by the second process.

[0095] Understandably, once the first process confirms that the second state data has been successfully received by the second process, the first process can release the second state data locally. This avoids excessive second state data occupies the first process, reducing resource consumption and increasing available resource space. Furthermore, if the second process fails to process the second state data, since the first process has already released it, the failed second state data is not allowed to remain in the first process; the second process can directly transfer the failed second state data from the second table to the fourth table without notifying the first process.

[0096] In some possible embodiments, the second process releases the processed second state data after returning it to the first process.

[0097] Understandably, after the second process finishes processing the second state data and returns the processed second state data to the first process, the processed second state data is released locally to reduce the resource consumption of the second process and increase the available resource space of the second process.

[0098] In some possible embodiments, after the second process transfers the failed second state data from the second table to the fourth table, it also releases the failed second state data.

[0099] Understandably, for second-state data that fails to be processed by the second process, after the second process deletes the second-state data from the second table and adds it to the fourth table, the second process can release the second-state data from its local storage. This is to prevent the failed second-state data from occupying the second process indefinitely, thereby reducing the resource consumption of the second process and increasing the available resource space of the second process.

[0100] Please see Figure 4 , Figure 4 A schematic diagram illustrating the data state transition during the timed task data processing provided in an embodiment of this application is shown below. Figure 4 The state transition diagram shown illustrates the timed task data processing method of this application embodiment:

[0101] S1, for data status types "0-Unprocessed", "1-In Transit", "2-Processed", "3-Error", set up corresponding database tables "First Table", "Second Table", "Third Table", and "Fourth Table";

[0102] S2, store the unprocessed raw data in the database DB, and record the raw data in the first table. The raw data is located at node P0, and the state of the data is represented by a tuple {0, {DB}} (that is, the data is in an unprocessed state, and a copy of the data is in DB).

[0103] S3, when the preset timed data processing task starts, the first process P1 executes action a, retrieves the unprocessed raw data from the database, the data arrives at node P1, and the data status is {0, {DB, P1}} (that is, the data is in an unprocessed state, and the data copy is in DB and process P1); if the first process P1 does not retrieve the raw data, it directly jumps to step S9.

[0104] S4, the first process P1 executes action b1, deleting the original data from the first table; then it executes action b2, adding the original data to the second table; in this way, the data travels from node P1 through P2 to P3, and the data state changes from {0, {DB, P1}} to {1, {P1}}, and then to {1, {DB, P1}} (that is, the data is in transit, and the copies of the data are in DB and P1).

[0105] S6, the first process P1 executes action c, sending the original data to the second process P2. The data arrives at node P4, and the data status is {1, {DB, P2}} (that is, the data is in transit, and the data copy is in DB and P2); if the first process does not receive the confirmation information returned by the second process P2, it jumps to step S10.

[0106] S7, the second process P2 receives the original data, processes the original data to obtain the processed data, and executes action d to send the processed data to the first process P1. The data is returned to node P3, and the data status is {1, {DB, P1}}. If the second process P2 fails to process the original data, it jumps to step S11.

[0107] S8, the first process P1 executes action e, deletes the processed data from the second table and adds it to the third table, and then releases the data. The data arrives at node P6, and the data status is {2, {DB}} (that is, the data is in the processed state, and the data copy is in DB).

[0108] S9. If the first process P1 does not obtain the original data, then execute action f, delete the original data from the first table and add the original data to the fourth table, and then release the original data; in this way, the data arrives from node P1 to P7, and the data state changes from {0, {DB, P1}} to {3, {DB}} (the data is in an error state, and the data copy is in DB).

[0109] S10, if the first process does not receive the confirmation information returned by the second process P2, then execute action g, delete the original data from the second table and add it to the fourth table, and then release the original data; in this way, the data moves from node P3 to P7, and the data state changes from {1, {DB, P1}} to {3, {DB}} (the data is in an error state, and the copy of the data is in DB).

[0110] S11, if the second process P2 fails to process the original data, then execute action h, delete the original data from the second table and add it to the fourth table, and then release the original data; in this way, the data moves from node P4 to P7, and the data state changes from {1, {DB, P2}} to {3, {DB}} (the data is in an error state, and a copy of the data is in DB).

[0111] The following section will further explain the timed task data processing method provided in the embodiments of this application, taking into account specific application scenarios.

[0112] The intelligent outbound calling robot platform needs to execute outbound calling tasks periodically. The platform regularly updates the list of customers requiring outbound calls, adding the customer data to table _0. When it's time to execute an outbound calling task, the intelligent outbound calling robot platform uses process P1 to retrieve the customer data to be called from table _0, and then transfers the customer data from table _0 to table _1. Process P1 sends the customer data to process P2, which then executes the outbound calling task. After process P2 completes the customer's outbound call, it returns the processed customer data to process P1. Process P1 then transfers the processed customer data from table _1 to table _2, indicating that the customer data in table _2 is the processed customer data. If an error occurs at any stage of the above process, the erroneous customer data is transferred to table _3, and table _3 records the erroneous customer data.

[0113] The method provided in this application embodiment ultimately achieves an ideal improvement in the efficiency of the platform in processing scheduled tasks. It should be noted that although this application embodiment increases the number of data tables, it does not increase the space complexity of the task, because the total amount of data in the database increases and decreases the same after each state change. Even if multiple add and delete operations are performed, only an equal amount of data is actually transferred from the first table to the second table. Apart from the DDL information of the data table itself, the program does not occupy any additional storage space.

[0114] Please see Figure 5 In some possible embodiments of this application, a timed task data processing apparatus 800 based on data status is also provided, which can implement the above-described timed task data processing method based on data status. The apparatus 800 includes:

[0115] The data acquisition module 801 is used to respond to the start of a preset timed data processing task and obtain first state data from the database through a first process according to a first table, wherein the first table is used to record the first state data;

[0116] The first data transfer module 802 is used to transfer the acquired first state data from the first table to the second table through the first process, so that the first state data is converted into second state data, wherein the second table is used to record the second state data;

[0117] The sending module 803 is used to send the second status data to the second process through the first process;

[0118] The processing module 804 is used to process the second state data based on a preset task through the second process, and return the processed second state data to the first process.

[0119] The second data transfer module 805 is used to transfer the processed second state data from the second table to the third table through the first process, so that the second state data is converted into third state data, wherein the third table is used to record the third state data.

[0120] In some embodiments, the apparatus 800 further includes a third data transfer module, configured to transfer the failed first state data to a fourth table when the first process fails to acquire the first state data, thereby converting the first state data into fourth state data, wherein the fourth table is used to record the fourth state data.

[0121] In some embodiments, the apparatus 800 further includes a fourth data transfer module, configured to transfer the failed second state data to a fourth table when the second process fails to process the second state data, thereby converting the second state data into fourth state data, wherein the fourth table is used to record the fourth state data.

[0122] In some embodiments, the device 800 further includes a confirmation module, configured to receive a confirmation message returned by the second process within a preset time interval, wherein the confirmation message indicates that the second process has received the second status data.

[0123] In some embodiments, the device 800 further includes a fifth data transfer module, which is used to transfer the second state data to a fourth table when no acknowledgment message is received within a preset time interval, so that the second state data is converted into fourth state data, wherein the fourth table is used to record the fourth state data.

[0124] In some embodiments, the apparatus 800 further includes a data release module, configured to determine the second state data that has been received by the second process based on the confirmation message, and the first process releases the second state data that has been received by the second process.

[0125] For example, the first data transfer module 802 is specifically used to: delete the first state data obtained from the first table through the first process, and add the first state data deleted from the first table to the second table.

[0126] For example, the second data transfer module 805 is specifically used to: delete the processed second state data from the second table through the first process; and add the second state data deleted from the second table to the third table.

[0127] It should be noted that the specific implementation of the multi-turn dialogue management device provided in this application is basically the same as the specific implementation of the timed task data processing method based on data status described above, and will not be repeated here.

[0128] This application also provides an electronic device, which includes a memory and a processor. The memory stores computer programs or instructions, and the processor executes the computer programs or instructions to implement the aforementioned timed task data processing method based on data state. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0129] Please see Figure 6 , Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device includes:

[0130] The processor 1010 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0131] The memory 1020 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010 using the timed task data processing method of the embodiments of this application.

[0132] The input / output interface 1030 is used to implement information input and output;

[0133] The communication interface 1040 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0134] Bus 1050 transmits information between various components of the device (e.g., processor 1010, memory 1020, input / output interface 1030, and communication interface 1040);

[0135] The processor 1010, memory 1020, input / output interface 1030 and communication interface 1040 are connected to each other within the device via bus 1050.

[0136] This application also provides a computer-readable storage medium storing a computer program or instructions that, when executed by a processor, implement the above-described timed task data processing method based on data state.

[0137] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0138] The data processing method, apparatus, device, and storage medium based on data status proposed in this application first respond to the start of a preset timed data processing task. A first process retrieves first status data from a database according to a first table, where the first table records the first status data. This concentrates unprocessed data in the first table, completely eliminating the potential timeout risk caused by excessive time consumption of the scheduled task as the total amount of data in the data table increases. Then, the first process transfers the retrieved first status data from the first table to a second table, converting the first status data into second status data. The second table records the second status data. The first process sends the second status data to a second process, thus concentrating the data already in transit in the process in the second table. The second process processes the second status data based on a preset task and returns the processed second status data to the first process. The first process then transfers the processed second status data from the second table to a third table, converting the second status data into third status data. The third table records the third status data. The solution in this application embodiment records data in different states through different database tables, avoiding the drawbacks of using flag fields to identify the processing status of data in the data tables, and reducing the time consumption generated when searching for unprocessed data in the data processing flow.

[0139] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0140] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0141] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0143] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0144] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: "a", "b", "c", "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0145] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0146] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0147] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0148] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0149] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A data state-based timing task data processing method, characterized by, The method comprises the following steps: In response to a preset timing data processing task starting, acquiring first state data from a database according to a first table by a first process, wherein the first table is used for recording the first state data; Transferring the acquired first state data from the first table to a second table by the first process, so that the first state data is converted into second state data, wherein the second table is used for recording the second state data; Sending the second state data to a second process by the first process; Processing the second state data based on a preset task by the second process, and returning the processed second state data to the first process; Transferring the processed second state data from the second table to a third table by the first process, so that the second state data is converted into third state data, wherein the third table is used for recording the third state data; The first process receives an acknowledgement message returned by the second process within a preset time interval, and the acknowledgement message indicates that the second process has received the second state data; When the acknowledgement message is not received within the preset time interval, the first process transfers the second state data to a fourth table, so that the second state data is converted into fourth state data, wherein the fourth table is used for recording the fourth state data.

2. The data state based timing task data processing method of claim 1, wherein, After acquiring the first state data from the first table of the database by the first process, the method further comprises: When the first process fails to acquire the first state data, transferring the failed first state data to a fourth table, so that the first state data is converted into fourth state data, wherein the fourth table is used for recording the fourth state data.

3. The data state based timing task data processing method of claim 1, wherein, After processing the second state data by the second process based on a preset task, the method further comprises: When the second process fails to process the second state data, transferring the failed second state data to a fourth table, so that the second state data is converted into fourth state data, wherein the fourth table is used for recording the fourth state data.

4. The data state based timing task data processing method of claim 1, wherein, After the first process receives the acknowledgement message returned by the second process within a preset time interval, the method further comprises: Determining the second state data that has been received by the second process according to the acknowledgement message; The first process releases the second state data that has been received by the second process.

5. The data state based timing task data processing method of claim 1, wherein, The transferring of the acquired first state data from the first table to the second table by the first process comprises: Deleting the acquired first state data from the first table by the first process; Adding the first state data deleted from the first table to the second table.

6. The data state based timing task data processing method of claim 1, wherein, The transferring of the processed second state data from the second table to the third table by the first process comprises: Deleting the processed second state data from the second table by the first process; Adding the second state data deleted from the second table to the third table.

7. A data state based timing task data processing apparatus, characterized by, The device comprises: a data acquisition module, configured to acquire first state data from a database according to a first table through a first process in response to a preset timing data processing task starting, wherein the first table is used to record the first state data; a first data transfer module, configured to transfer the acquired first state data from the first table to a second table through the first process, so that the first state data is converted into second state data, wherein the second table is used to record the second state data; a sending module, configured to send the second state data to a second process through the first process; a processing module, configured to process the second state data based on a preset task through the second process, and return the processed second state data to the first process; a second data transfer module, configured to transfer the processed second state data from the second table to a third table through the first process, so that the second state data is converted into third state data, wherein the third table is used to record the third state data; a confirmation module, configured to receive an acknowledgement message returned by the second process within a preset time interval through the first process, the acknowledgement message indicating that the second process has received the second state data; a fifth data transfer module, configured to transfer the second state data to a fourth table through the first process when the acknowledgement message is not received within the preset time interval, so that the second state data is converted into fourth state data, wherein the fourth table is used to record the fourth state data.

8. An electronic device, comprising: The computer readable storage medium stores computer programs or instructions, and the processor executes the computer programs or instructions to implement the data state-based timing task data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer programs or instructions, and the processor executes the computer programs or instructions to implement the data state-based timing task data processing method according to any one of claims 1 to 6.

Images