Data processing method and device and related equipment
By scheduling and re-flashing the service payment data at the data processing time point, the problem of cross-day association of payment service data in the prior art is solved, and data integrity and state update accuracy are improved.
Patent Information
- Application Number
- CN202410075178.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-18
- Publication Date
- 2025-07-18
AI Technical Summary
In the prior art, the cross-day scheduling processing of service payment data makes it difficult to associate the data of the same payment service, affecting data integrity.
By scheduling and processing the service payment data in the first time period at the data scheduling time point when the data processing time reaches the second time period, the first service status data is obtained, and the data re-sweeping data and the second service payment data are re-sweeping based on the data re-sweeping link to update the service status.
The data association of payment service across multiple time periods is realized, the data integrity of payment service is improved, and the accurate update of payment service status is ensured.
Smart Images

Figure CN120337304A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of data processing, and in particular to data processing methods, devices, and related equipment. Background Art
[0002] In some business systems (such as business payment systems), it is usually necessary to schedule the business data (such as business payment data) that occurs in the system for writing into the database. Currently, the processing of business payment data is carried out in a T+1 manner, that is, the data on the Tth day is scheduled on the (T + 1)th day.
[0003] However, the inventor found in the practice process that in some payment operations, the business data of a single operation (such as a red envelope transfer operation) needs to be scheduled across days. That is, the associated data of a single operation (such as the red envelope receiving information of a red envelope transfer operation) may not appear on the same day, but on some future days. Moreover, since the business data generated each day is processed independently, it is difficult to associate the business data of a single operation. For example, in some payment scenarios, user A initiates a payment operation on the Tth day, and user B can actually receive it on the (T + 2)th day. Therefore, when scheduling the data generated on the previous day (i.e., the Tth day) on the (T + 1)th day, the receiving information for this payment operation that is actually generated on the next day (i.e., the (T + 2)th day) cannot be obtained. This means that the current technical solution of using the T+1 method to schedule the data generated on the previous day (i.e., the Tth day) is difficult to associate the data for the same payment operation, making it difficult to ensure the data integrity of the payment operation. Summary of the Invention
[0004] Embodiments of this application provide a data processing method, device, and related equipment, which can perform data rebrushing processing on business payment data, helping to improve the data integrity of payment operations.
[0005] One aspect of the embodiments of this application provides a data processing method, the method including:
[0006] When the data processing time reaches the data scheduling time point within the second time period, through the data calculation node indicated by the business payment system, schedule the first business payment data generated by the first business object within the first time period to obtain first business status data, and store the first business status data in the business payment database; the first time period is the previous time period of the second time period; the second time period belongs to the first sliding processing time window corresponding to the first time period; the first business status data is used to represent that the business status of the payment operation initiated by the first business object is the first business processing status;
[0007] When the data processing time reaches the data rebrushing time point of the data within the third time period in the first sliding processing time window, based on the data calculation node associated with the payment service, determine the data rebrushing link for changing the first service processing state; the third time period is any time period in the first sliding processing time window that is later than the second time period;
[0008] Perform data rebrushing processing on the first service payment data and the second service payment data based on the data rebrushing link to obtain second service status data; the second service payment data includes service payment data associated with the payment service generated by the associated object associated with the first service object during the second time period to the third time period; the second service status data is used to characterize that the service status of the payment service is the second service processing state;
[0009] When updating the first service status data in the service payment database based on the second service status data, update the service status of the payment service from the first service processing state to the second service processing state.
[0010] One aspect of the embodiments of the present application provides a data processing device, the device includes:
[0011] A data scheduling module, configured to, when the data processing time reaches the data scheduling time point within the second time period, perform data scheduling processing on the first service payment data generated by the first service object during the first time period through the data calculation node indicated by the service payment system, obtain first service status data, and store the first service status data in the service payment database; the first time period is the previous time period of the second time period; the second time period belongs to the first sliding processing time window corresponding to the first time period; the first service status data is used to characterize that the service status of the payment service initiated by the first service object is the first service processing state;
[0012] A rebrushing link determination module, configured to, when the data processing time reaches the data rebrushing time point within the third time period in the first sliding processing time window, determine a data rebrushing link for changing the first service processing state based on the data calculation node associated with the payment service; the third time period is any time period in the first sliding processing time window that is later than the second time period;
[0013] A data rebrushing module, configured to perform data rebrushing processing on the first service payment data and the second service payment data based on the data rebrushing link to obtain second service status data; the second service payment data includes service payment data associated with the payment service generated by an associated object associated with the first service object during the second time period to the third time period; the second service status data is used to represent that the service status of the payment service is a second service processing status;
[0014] A data update module, configured to update the service status of the payment service from the first service processing status to the second service processing status when updating the first service status data in the service payment database based on the second service status data.
[0015] On the one hand, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which is suitable for being loaded and executed by a processor so that a computer device having the processor executes the method provided by the embodiment of the present application.
[0016] On the one hand, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device executes the method provided by the embodiment of the present application.
[0017] In an embodiment of the present application, a corresponding sliding processing time window (i.e., the first sliding processing time window) can be determined for a time period (such as the first time period). When the data scheduling time point of the data in the next time period (i.e., the second time period) after the data processing time reaches the first time period arrives, data scheduling processing can be performed based on the business payment data generated by the first business object within the first time period (i.e., the first business payment data) to obtain the first business status data, so as to characterize that the business status of the payment business is the first business processing status. Furthermore, when the data processing time reaches the data rebrushing time point within the sliding processing time window, data rebrushing processing can be performed based on the first business payment data and the second business payment data to obtain the second business status data for characterizing that the business status of the payment business is the second business processing status, thereby updating the business status of the payment business. Thus, by performing data rebrushing processing in the time periods within the sliding processing time window, the business payment data in multiple time periods can be associated, and based on the business payment data in multiple time periods, the business status of the payment business can be determined, and the business status of the payment business can be continuously updated during the processing of the payment business, recording the state changes of a payment business from initiation to completion that span multiple time periods together, improving the data integrity of the payment business. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts.
[0019] Figure 1 is a schematic structural diagram of a data processing system provided by an embodiment of the present application;
[0020] Figure 2 is a schematic scenario diagram of a data processing method provided by an embodiment of the present application;
[0021] Figure 3 is a schematic flowchart of a data processing method provided by an embodiment of the present application;
[0022] Figure 4 is a schematic scenario diagram of a payment business provided by an embodiment of the present application;
[0023] Figure 5 is a schematic diagram of the hierarchical effect of a data calculation node of a payment business provided by an embodiment of the present application;
[0024] Figure 6 is a schematic diagram of the effect of a calculation node topology diagram provided by an embodiment of the present application;
[0025] Figure 7 It is a schematic diagram of the effect of business payment data for data rebrushing provided by an embodiment of the present application;
[0026] Figure 8 It is a schematic flowchart of a data processing method provided by an embodiment of the present application;
[0027] Figure 9 It is a schematic diagram of the effect of a business payment data provided by an embodiment of the present application;
[0028] Figure 10 It is a schematic diagram of the effect of a data import process provided by an embodiment of the present application;
[0029] Figure 11 It is a schematic diagram of the resource occupancy situation of data processing provided by an embodiment of the present application;
[0030] Figure 12 It is a schematic diagram of the effect of a data backbrushing link provided by an embodiment of the present application;
[0031] Figure 13 It is a sliding schematic diagram of a sliding processing time window provided by an embodiment of the present application;
[0032] Figure 14 It is a schematic diagram of the determination of business payment data provided by an embodiment of the present application;
[0033] Figure 15 It is a schematic diagram of the determination of business payment data provided by an embodiment of the present application;
[0034] Figure 16 It is a schematic diagram of the determination of business payment data provided by an embodiment of the present application;
[0035] Figure 17 It is a schematic diagram of the effect of a query page for the processing status of data provided by an embodiment of the present application;
[0036] Figure 18 It is a schematic diagram of the backbrushing effect provided by an embodiment of the present application;
[0037] Figure 19 It is a schematic structural diagram of a data processing device provided by an embodiment of the present application;
[0038] Figure 20 It is a schematic structural diagram of a computer device provided by an embodiment of the present application. Detailed implementation manners
[0039] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.
[0040] Please refer to Figure 1 , Figure 1 which is a schematic structural diagram of a data processing system provided by an embodiment of the present application. As Figure 1 shown, the data processing system may include terminal devices (such as device 11a, device 12a, device 13a) and a server 200a. It can be understood that Figure 1 the numbers of terminal devices and servers in
[0041] are merely illustrative. According to actual needs, there can be any number of terminal devices and servers. The terminal devices (such as device 11a, device 12a, device 13a) can communicate with the server through a network (that is, a medium providing a communication link through wired, wireless communication links, fiber optic cables, etc.), and then perform data transmission.
[0042] It can be understood that a client can run on the terminal device (such as device 12a), and the client can be a program that provides local services for users (also known as business objects, operation objects). The server 200a can be the server corresponding to the client, and a program for providing resources, service data, and other services can run in the server 200a. It can be understood that the client running on the terminal device can also be called an application client, a business client, etc. For example, the client running on the terminal device can be a client for providing payment services, and then the user can initiate a payment transaction on the terminal device.
[0043] It can be understood that the embodiments of the present application can be applied to computer devices, and the computer device can be the above Figure 1 server, or other devices used for the above Figure 1A data processing device that processes business data within a data processing system. The data processing device can be a server or a terminal device. Among them, the terminal device can include, but is not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, smart speakers, smart home appliances, etc., and there is no limitation here. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, and there is no limitation here.
[0044] Please refer to Figure 2 , Figure 2 is a schematic diagram of the scenario of a data processing method provided by an embodiment of the present application. As Figure 2 shown, the data processing time in the system can be divided into multiple time periods, such as time periods T1, T2, and T3. As Figure 2 shown, the business object can generate business payment data Y1 within time period T1 (such as Figure 2 shown as 201a in
[0045] ). This time period T1 can be associated with a corresponding sliding processing time window, for example, shown as sliding processing time window 203a. In this sliding processing time window 203a, it can include time periods T2 and T3.
[0046] Furthermore, when the data processing time reaches the data scheduling time point t1 within time period T2, the business payment data Y1 can be subjected to data scheduling processing to obtain business status data Z1 (step S204a), and then the determined business status data Z1 can be stored in the business payment database 206a.
[0046] Furthermore, when the data processing time reaches the data rebrushing time point t2 within time period T3, the business payment data Y1 and the business payment data Y2 can be subjected to data rebrushing processing to obtain business status data Z2 (step S205a). Among them, the business payment data Y2 (such as Figure 2 shown as 202a in
[0047] In the embodiments of the present application, a corresponding sliding processing time window (i.e., the first sliding processing time window) can be determined for a time period (such as the first time period). When the data scheduling time point of the data in the next time period (i.e., the second time period) after the data processing time reaches the first time period is reached, data scheduling processing can be performed based on the business payment data (i.e., the first business payment data) generated by the first business object in the first time period to obtain the first business status data, so as to represent that the business status of the payment business is the first business processing status. Furthermore, when the data processing time reaches the data rebrushing time point within the sliding processing time window, data rebrushing processing can be performed based on the first business payment data and the second business payment data to obtain the second business status data for representing that the business status of the payment business is the second business processing status, thereby updating the business status of the payment business. Thus, by performing data rebrushing processing in the time periods within the sliding processing time window, the business payment data in multiple time periods can be associated, so as to determine the business status of the payment business based on the business payment data in multiple time periods, continuously update the business status of the payment business during the processing of the payment business, record the state changes of a payment business throughout the process from initiation to end spanning multiple time periods together, and improve the data integrity of the payment business.
[0048] In addition, this special periodic sliding window (i.e., the sliding processing time window) can also be combined with the jump time window supplement. This rebrushing mechanism can better solve the problem of missing penetration association caused by time drift and business cross-day drift. By setting the execution dependency of the incremental task and setting the rebrushing start time point at a time when the cluster computing resources are relatively idle at noon, without affecting the operation of the incremental instance, the utilization rate of the cluster computing resources is balanced at the same time, and it is avoided that all resources appear at the early morning peak, affecting the operation of the cluster.
[0049] It can be understood that it should be noted that before and during the process of collecting the relevant data of the user in the present application, a prompt interface, a pop-up window or voice prompt information can be displayed. The prompt interface, the pop-up window or the voice prompt information is used to prompt the user that their relevant data is currently being collected, so that the present application only starts to execute the relevant steps of obtaining the relevant data of the user after obtaining the confirmation operation of the user on the prompt interface or the pop-up window. Otherwise (i.e., when the confirmation operation of the user on the prompt interface or the pop-up window is not obtained), the relevant steps of obtaining the relevant data of the user are ended, that is, the relevant data of the user is not obtained. In other words, all the user data collected by the present application is collected with the consent and authorization of the user, and the collection, use and processing of the relevant user data need to comply with the relevant laws, regulations and standards of the relevant region.
[0050] It can be understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided by the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, as is known to those of ordinary skill in the art, with the evolution of system architectures and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.
[0051] Further, please refer to Figure 3 , Figure 3 which is a schematic flowchart of a data processing method provided by an embodiment of the present application. This method can be executed by a computer device, such as the server 200a in the above Figure 1 . This method can at least include the following steps S101 - step S104.
[0052] S101. When the data processing time reaches the data scheduling time point within the second time period, through the data calculation node indicated by the business payment system, perform data scheduling processing on the first business payment data generated by the first business object within the first time period to obtain first business status data, and store the first business status data in the business payment database; the first time period is the previous time period of the second time period; the second time period belongs to the first sliding processing time window corresponding to the first time period; the first business status data is used to represent that the business status of the payment business initiated by the first business object is the first business processing status.
[0053] Among them, the data processing time may refer to the timing information used for data processing. For example, the data processing time may be the system time. In computer science and computer programming, the system time represents the computer system's concept of the passage of time.
[0054] It can be understood that in the present application, the data processing time can be divided into multiple time periods, and the cycle lengths of each time period are the same. For example, each time period is 1 day. For example, from 0:00 on January 1st to 0:00 on January 2nd is a time period, and from 0:00 on January 2nd to 0:00 on January 3rd is the next time period, and so on. Another example is that each time period is 12 hours. For example, from 0:00 on January 1st to 12:00 on January 1st is a time period, and from 12:00 on January 1st to 0:00 on January 2nd is the next time period. It can be understood that the division method of the time period can be determined according to actual needs and is not limited here.
[0055] It can be understood that the second time period here can be any time period among the time periods divided based on the data processing time. The first time period is the previous time period of the second time period. For example, the second time period is the time period T2 in the above Figure 2 , and the first time period is the above Figure 2The time period T1 therein. For example, each time period is 1 day. When the time period corresponding to the second time period is January 2nd, the first time period can be the time period corresponding to January 1st, and no limitation is made here.
[0056] It can be understood that the first sliding processing time window corresponding to the first time period can be the time window composed of the time periods used to process the business payment data (i.e., the first business payment data) within the first time period. It should be understood that the second time period data and this first sliding processing time window. It can be understood that within this first sliding processing time window, in addition to the second time period, at least one other time window can also be included. It should be understood that for the payment business data within each time period, a corresponding sliding processing time window can be determined to determine the business status of the payment business initiated within the corresponding time period at the corresponding time point within the sliding processing time window.
[0057] It should be understood that the sliding processing time window (such as the first sliding time window) can be determined based on the processing duration of the payment business to be processed. Generally speaking, the sliding processing time window needs to be greater than or equal to the processing duration of the payment business. It can be understood that the sliding processing time window can also be referred to as the sliding processing time window, the sliding time window, etc., and no limitation is made here.
[0058] For example, please refer to Figure 4 , Figure 4 is a schematic diagram of a payment business scenario provided by an embodiment of the present application. As Figure 4As shown, a transfer service needs to transfer funds to an intermediate object on the day after the initiating object of the service initiates the transfer service (such as transferring to a payment platform party, which can also be referred to as a merchant), and then on the third day, the intermediate object transfers the funds to the receiving object. That is, the current fund flow pattern is from an individual to an enterprise (i.e., C2B), and then from the enterprise to an individual (i.e., B2C), and it is a business scenario where the user selects to enable the delayed credit function. For example, business object A initiates a transfer service on the Tth day, and the assets indicated by this transfer service will be transferred to the intermediate object on the (T + 1)th day, and on the (T + 2)th day, the intermediate object transfers the funds to business object B to complete the entire transfer process of this service. Additionally, if the receiving object (such as business object B) does not confirm the receipt of this payment service within a certain period of time (such as 24 hours), the assets transferred by this service will be returned by the intermediate object to business object A. Therefore, the entire processing process of this payment service will involve 3 days (i.e., the Tth day, the (T + 1)th day, and the (T + 2)th day). Furthermore, the size of the sliding processing time window for this type of transfer service can be determined to be 3 days, that is, 3 time periods. Since the business payment data in the previous time period can be processed in the next time period, the sliding processing time window corresponding to the Tth day can include: the (T + 1)th day, the (T + 2)th day, and the (T + 3)th day. Similarly, the sliding processing time window corresponding to the (T + 1)th day can include: the (T + 2)th day, the (T + 3)th day, and the (T + 4)th day.
[0059] The data scheduling time point (which can also be simply referred to as the scheduling time point) can be the time point used to trigger data scheduling processing. The data scheduling processing can refer to the operation of coordinating and arranging system tasks (i.e., the tasks corresponding to the data calculation nodes indicated by the business payment system). For example, the data scheduling processing can be the processing operation used to indicate the execution of system tasks such as ETL (Extract, Transform, and Load) tasks, scenario partitioning tasks, data summarization tasks, data import tasks, etc. For example, the data scheduling time point can be the start time point of the second time period (i.e., the end time point of the first time period). If the first time period is January 1st and the second time period is January 2nd, then the data scheduling time point within the second time period can be 0:00 on January 2nd (i.e., 24:00 on January 1st). Furthermore, after a time period ends, the business payment data generated within that time period can be quickly subjected to data scheduling processing to ensure the timeliness of the data scheduling processing, which helps to carry out business operations (such as data query) based on the results obtained from the data scheduling processing. It can be understood that there can be corresponding data scheduling time points within each time period for data scheduling processing based on the business payment data generated by the business payment system in the previous time period. For example, when the data processing time reaches the data scheduling time point within the 2nd time period (such as 0:00 on January 2nd), the business payment data generated within the 1st time period (such as January 1st) can be subjected to data scheduling processing. When the data processing time reaches the data scheduling time point within the 3rd time period (such as 0:00 on January 3rd), the business payment data generated within the 2nd time period (such as January 2nd) can be subjected to data scheduling processing.
[0060] Among them, the data calculation node can be used to indicate the calculation logic for data processing. A data processing node is equivalent to a link in the process of data scheduling processing to perform corresponding data processing operations. It can be understood that the data calculation nodes used for data scheduling processing can be determined according to the actual needs of the payment business, and no limitation is made here.
[0061] For example, please refer to Figure 5 , Figure 5 which is a schematic diagram of the hierarchical effect of the data calculation nodes of a payment business provided by an embodiment of the present application. As Figure 5As shown, the data calculation node may include a data start node (shown as 501a), a scenario node (shown as 502a), a verification node (shown as 503a), a summary node (shown as 504a), an import node (shown as 505a), and so on. Among them, the data start node can be used to obtain the production-side data source 500a from the business payment system. The production-side data source can be the business payment data generated during the operation of the business payment system. Then, based on the production-side data source, the data source of the ODS layer (i.e., the operational data layer) is determined. The data calculation node that determines the data source of the OSD layer can be the start node 501a. Further, the business data at the operation layer can be split by scenario to obtain business data under multiple scenarios, such as business data under scenarios 1, 2, 3, etc. It can be determined that the processing logic of the business data under each scenario can correspond to a data calculation node, which can be called a scenario node. For example, the scenario node here can be a scenario node for indicating data division by sub-account type, a scenario node for data division by sub-transaction method, a scenario node for data division by sub-business scenario, and a scenario node for obtaining the intermediate layer process table. In addition, the business data under each scenario can be verified. The data calculation node for verifying the business data under each scenario can be called a verification node, such as the verification nodes corresponding to verification 1, verification 2, and verification 3. For example, the verification node here can be a verification node for detecting garbled characters, a verification node for detecting the exception rate, and a verification node for detecting integrity.
[0062] Further, the business data under each scenario can be summarized by scenario, that is, one or more data tables can be obtained based on the summary of the business data under each scenario. The data calculation node used for scenario summary can be called a summary node (or data summary node). It should be understood that the summary node can include the summary nodes corresponding to obtaining data tables such as total table 1, total table 2, total table 3, etc. For example, the summary node can be a summary node for obtaining the summary table of user C (i.e., the summary table of business data associated with user C), a summary node for obtaining the summary table of user-to-card (i.e., the summary table of business data for user transfers to bank cards), a summary node for obtaining the summary table of merchant B (i.e., the summary table of business data associated with merchant B), a summary node for obtaining the summary table of merchant C (i.e., the summary table of business data associated with merchant C), and so on. It can be understood that the data obtained through summary by the summary node can be stored in the data warehouse. For example, the data warehouse can be a TDW data warehouse.
[0063] Further, the summarized data table can be imported. The data calculation node for importing the summarized data table can be referred to as an import node (or data import node). Herein, the import node can include an import node 505a to convert the summarized data table into intermediate table 1 and intermediate table 2, and import the summarized data table into the database cluster 506a based on intermediate table 1 and intermediate table 2. For example, the database cluster can be an HBase (a kind of database) cluster, so as to obtain main table 1 and main table 2 stored in the database cluster 506a. The main table 1 and main table 2 can be used for business objects to query business status data. Optionally, the import node herein can also include a data calculation node for directly importing the summarized database into the database backup cluster 507a. For example, the database backup cluster can be an HBase (a kind of database) cluster, so as to back up the summarized data table. Optionally, the import node herein can also include an import node for importing the summarized data table into the cluster management application 508a to manage the data stored in the database cluster through the cluster management application.
[0064] Further, to facilitate the determination of data calculation nodes, a calculation node topology graph can be constructed based on each data calculation node in the business payment system, so as to quickly determine the data calculation node for processing the business payment data of the payment business subsequently. It is determined that the calculation node topology graph includes calculation nodes and connection edges; a calculation node is used to indicate the calculation logic for data processing; a connection edge is determined based on the calculation order between the two connected calculation nodes. It should be understood that the calculation node topology graph can be a directed graph, so that the calculation order between data calculation nodes can be represented by the direction of the connection edge. The calculation order can be used to indicate the sequence of the calculation processes of the calculation nodes. If it is necessary to call calculation node A for calculation first and then call calculation node B for calculation, then in the calculation node topology graph, the direction of the connection edge between calculation node A and calculation node B is from A to B. It should be understood that if there is no direct calculation order relationship between two calculation nodes, there is no connection edge between them.
[0065] For example, please refer to Figure 6 , Figure 6 is a schematic diagram of the effect of a calculation node topology graph provided by an embodiment of the present application. As Figure 6 shown, in this (as shown in 600a in Figure 6 ), it can include multiple calculation nodes and the connection edges between the calculation nodes. Corresponding to the hierarchical schematic diagram of the data calculation nodes of the payment business shown in the above Figure 5 , in this calculation node topology graph (as Figure 6As shown by 600a in [context], it may include data calculation nodes such as a starting node (e.g., starting node 601a), a scenario node (e.g., scenario node 602a), a summary node (e.g., summary node 603a), and an import node 604a. It can be seen that the connection edge between the starting node 601a and the scenario node 602a points from the starting node 601a to the scenario node 602a, indicating that the starting node 601a is calculated first and then the scenario node 602a. Similarly, the connection edge between the scenario node 602a and the summary node 603a points from the scenario node 602a to the summary node 603a, indicating that the scenario node 602a is calculated first and then the summary node 603a. The connection edge between the summary node 603a and the import node 604a points from the summary node 603a to the import node 604a, indicating that the summary node 603a is calculated first and then the import node 604a. The connection edge between the scenario node 605b and the verification node 605a points from the scenario node 605b to the verification node 605a, indicating that the scenario node 605b is calculated first and then the verification node 605a.
[0066] It can be understood that the first business payment data can be the business payment data generated by the first business object within the first time period, or it can be described as the business payment data within the first time period. The first business object can be the business object that initiates the payment business within the first time period, and the number of such first business objects can be one or more, which is not limited here. Among them, the payment business can be a business based on the business payment system, and the business payment data can be the data generated when initiating the payment business. For example, the payment business can be a business of transferring funds from one business object (i.e., the initiating object) to another business object (i.e., the receiving object) (i.e., the transfer business), then the business payment data of this payment business (such as the first business payment data) can be the water bill for the transfer, and the water bill can record information such as the initiation time of the transfer business, the identifier of the initiating object, the transfer type, and the transfer amount. Another example is that the payment business can be a business of a business object (i.e., the repayment object) making a repayment (i.e., the repayment business), then the business payment data of this payment business (such as the first business payment data) can be the water bill for the repayment, and the water bill can record information such as the initiation time of the repayment business, the identifier of the repayment object, and the repayment amount.
[0067] Among them, the first service status data can be used to represent that the service status of the payment service initiated by the first service object is the first service processing status. Among them, the first service processing status can be the service status determined based on the service payment data (i.e., the first service payment data) within the first time period, that is, the first service processing status is used to represent the service status of the payment service at the end of the first time period. Among them, the service status can be used to indicate the status of the payment service. It should be understood that during the processing of a payment service, multiple processing links may be involved, that is, a payment service may correspond to multiple service statuses. For example, in a transfer service, the service statuses include the un-received status, the received status, etc. In a red envelope service, the service statuses include the unclaimed status, the claimed status, the returned status, etc. The first service status data can be used to record the data of the service status of the payment service. It can be understood that the number of the first service objects can be one or more, and the number of payment services initiated by one first service object can also be one or more, which is not limited here. Further, it can be understood that the first service status data can include the service status data of one or more payment services to represent that the service status of one or more payment services is the first service processing status.
[0068] It can be understood that in the first service status data, for the service status data of each payment service, it can include the service entity information and N service associated information corresponding to the service entity information, where N is a positive integer. The service entity information can be the information of the initiator of the payment service. For example, it includes the identifier of the service object that initiates the payment service, the initiation time of the payment service, the asset amount (also known as the asset quantity) involved in the payment service, the service type of the payment service, the transaction identifier of the payment service, etc. The service associated information can be the information associated during the processing of the payment service. For example, the associated information can be used to indicate the information of the recipient of the payment service. For example, in a transfer service, it includes the identifier of the recipient object, the receipt time, the received amount, etc. If the payment service involves the fund transfer of multiple objects, the associated information can include the information of the objects involved in each asset transfer during the processing of the payment service. For example, user A first transfers the asset to merchant B (i.e., the transfer object), and then merchant B transfers the asset transferred by user A to user B. Then the associated information can include the information of merchant B (such as the receipt time, the received amount, the identifier of merchant B, etc.), and the information of object B (such as the receipt time, the received amount, the identifier of object B, etc.). It should be understood that if the payment service initiated within the first time period is not processed within the first time period, for example, in the red envelope service initiated within the first time period, no object claims it, then the associated information corresponding to this payment service can be a null value.
[0069] For example, after the business object A (corresponding to the first business object) initiates a transfer business to transfer funds to the business object B on the Tth day, if the business object B does not receive the assets indicated by the transfer business on the Tth day, the first business status data determined based on the payment business data on the Tth day may only include information about the initiator, such as the initiation time of the transfer business initiated by the business object A and the identifier of the business object A. The information about the recipient of this transfer business (such as the receipt time, the identifier of the business object B, etc.) is a null value, indicating that the business status of this transfer business (a type of payment business) is the un-received status. If the business object B has received the assets indicated by the transfer business on the Tth day, the first business status data may include information about the initiator, such as the initiation time of the transfer business initiated by the business object A and the identifier of the business object A, as well as the information about the recipient of this transfer business (such as the receipt time, the identifier of the business object B, etc.), indicating that the business status of this transfer business (a type of payment business) is the received status.
[0070] Another example is that the business object A initiates a red envelope business (a type of payment business) on the Tth day. If no object receives the red envelope on the Tth day, in the first business status data determined based on the payment business data on the Tth day on the (T + 1)th day, only the information about the initiator of the red envelope business is included, and the information about the recipient is a null value, indicating that the business status of this red envelope business is the un-received status. If there is an object B that receives the red envelope on the Tth day but does not receive it all, in the first business status data determined based on the payment business data on the Tth day on the (T + 1)th day, the information about the initiator of the red envelope business is included, as well as the information about object B receiving the red envelope (such as the amount received, the time, the identifier of object B, etc.), indicating that the business status of this red envelope business is the status of being received by one business object. If there are objects B, C, etc. that receive the red envelope on the Tth day and the amount corresponding to the red envelope business is received completely, in the first business status data determined based on the payment business data on the Tth day on the (T + 1)th day, the information about the initiator of the red envelope business is included, as well as the information about objects B, C, etc. receiving the red envelope, indicating that the business status of this red envelope business is the status of being received completely.
[0071] Among them, the business payment database can be a database for storing the first business status data. For example, the business payment database can be an HBase (a type of database) database. It can be understood that in the business payment database, the first business status data can be stored based on the first data format. When storing the first business status data in the business payment database based on the first data format, a main key value information can be determined for each main body field information in the business main body information of each payment business, and an associated key value information can be determined for each of the N business association information. One associated key value information can be used to store at least one associated field information in one business association information. Among them, the main body field information can be the information of the fields in the business main body information. For example, in the business main body information of a transfer business, the initiation time of the business is a field information, the identifier of the business object initiating the transfer business is a field information, the transfer amount is a field information, and so on. There is no limitation here. The associated field information can be the information of the fields in the business association information. For example, in the business association information of a transfer business, the identifier of the recipient object of the transferred asset is a field information, the receipt time is a field information, and so on. There is no limitation here. Among them, the main key value information can be the key value pair information determined based on the main body field information. One main body field information can correspond to one main key value information. For example, if the initiation time of a payment business is t1, then in the main key value information for the field information of the initiation time, the key is the initiation time and the value is t1. Among them, the associated key value information can be the key value pair information determined based on the business association information. One business association information corresponds to one associated key value information. In other words, multiple field information in the business association information can be determined as the value in a key value pair.For example, in the business association information of a red envelope business, it may include the associated field information 1 for business object B to receive the red envelope and the associated field information 2 for business object C to receive the red envelope. Among them, the associated field information 1 may include the field information of the identifier r1 of business object B, the field information of the received amount j1, and the field information of the time t2 when business object B receives the red envelope. Then, an associated key value information can be determined for the associated field information 1. The key is a string used to indicate that business object B receives the red envelope (such as a string for marking red envelope receipt + the identifier of business object B), and the value is ("identifier": "r1", "received amount": "j1", "receiving time": "t2"); Similarly, the associated field information 2 may include the field information of the identifier r2 of business object C, the field information of the received amount j2, and the field information of the time t3 when business object C receives the red envelope. Then, an associated key value information can be determined for the associated field information 2. The key is a string used to indicate that business object C receives the red envelope (such as a string for marking red envelope receipt + the identifier of business object C), and the value is ("identifier": "r2", "received amount": "j2", "receiving time": "t3").
[0072] S102. When reaching the data rebrushing time point of the data within the third time period in the first sliding processing time window during the data processing time, determine the data backhaul link for changing the first business processing status based on the data calculation node associated with the payment business; the third time period is any time period in the first sliding processing time window that is later than the second time period.
[0073] Among them, the third time period can be the time period for changing the first business processing status in the first sliding processing time window, that is, the time period for updating the business status of the payment business initiated within the first time period. It should be understood that this third time period is later than the second time period, and the number of this third time period can be one or more, specifically determined according to the actually determined sliding time window. When the number of the third time periods is multiple, each time period can be called a sub-time period under the third time period, such as the first sub-time period and the second sub-time period. For example, the sliding processing time window corresponding to the Tth day (i.e., the first time period) may include: the (T + 1)th day, the (T + 2)th day, the (T + 3)th day; among them, the (T + 1)th day can be the second time period, and the third time period can include two sub-time periods, the (T + 2)th day and the (T + 3)th day. Another example, the sliding processing time window corresponding to the Tth day (i.e., the first time period) may include: the (T + 1)th day, the (T + 2)th day; among them, the (T + 1)th day can be the second time period, and the third time period can include the (T + 2)th day.
[0074] Among them, the data rebrushing time point (which can also be simply referred to as the rebrushing time point) can be the time point for rebrushing the data of the first business processing state (i.e., changing the first business processing state). In other words, when the data processing time reaches the data rebrushing time point within the third time period, data rebrushing can be performed. Among them, data rebrushing (also known as data backbrushing, data rebrushing processing, data backbrushing processing, etc.) can be used to indicate the business payment data based on the first time period to the previous time period of the time period where the data rebrushing time point is located (including the first time period and the previous time period of the third time period where the data rebrushing time point is located, that is, excluding the time period where the data rebrushing time point is located), and re-determine the business state of the payment business initiated by the first business object within the first time period (i.e., perform data rebrushing). That is, based on the first business payment data and the business payment data of the previous time period from the second time period to the time period where the data rebrushing time point is located (including the second time period and excluding the third time period where the data rebrushing time point is located), if the third time period where the data rebrushing time point is located is the next time period of the second time period, then the second time period is the previous time period of the third time period where the data rebrushing time point is located, that is, data rebrushing is performed based on the first business payment data and the business payment data within the second time period.
[0075] It can be understood that the number of such data rebrushing time points can be determined according to the number of third time periods. If the third time period is a single one, then the data rebrushing time point can be a time point within this single third time period; if the number of third time periods is multiple, then each sub-time period within the third time period can contain a data rebrushing time point. Furthermore, when the data processing time reaches each data rebrushing time point, data rebrushing can be performed once.
[0076] Among them, when performing data rebrushing, the first service payment data and the second service payment data can be processed for data rebrushing based on data calculation nodes associated with the payment service. Among them, the second service payment data can include service payment data associated with the payment service generated by an associated object associated with the first service object during the second time period to the third time period. Specifically, it can be the service payment data (including the second time period and excluding the third time period where the data rebrushing time point is located) from the second time period to the previous time period of the time period where the data rebrushing time point is located. It should be understood that if the number of third time periods is single, the third time period where the data rebrushing time point is located is the next time period of the second time period, then the second service payment data is the service payment data within the second time period. If the number of third time periods is multiple, that is, the third time period can include multiple sub-time periods, then when the data processing time reaches the data rebrushing time point of the first sub-time period (i.e., the next time period of the second time period), the second service payment data is the service payment data within the second time period; when the data processing time reaches the data rebrushing time point of the second sub-time period (i.e., the next sub-time period of the first sub-time period), the second service payment data can include the service payment data within the second time period and the service payment data within the first sub-time period.
[0077] Among them, the data backbrushing link can be a link composed of data calculation nodes used for data rebrushing. The data backbrushing link can include the above-mentioned data calculation nodes and can indicate the calculation order between each data calculation node. In other words, the data backbrushing calculation link can be used to indicate to call data nodes for data backbrushing according to a certain calculation order. It should be understood that the data backbrushing link can be used to change the first service processing status, that is, it can be used to update the first service processing status (i.e., data backbrushing).
[0078] S103. Perform data rebrushing processing on the first service payment data and the second service payment data based on the data backbrushing link to obtain second service status data; the second service payment data includes service payment data associated with the payment service generated by an associated object associated with the first service object during the second time period to the third time period; the second service status data is used to characterize that the service status of the payment service is the second service processing status.
[0079] Among them, the second service status data can be used to represent that the service status of the payment service initiated by the first service object is the second service processing status. Among them, the second service processing status can be the service status determined based on the first service payment data and the second service payment data. It should be understood that this second service status can be the same as the first service processing status or different from the first service processing status, which is specifically determined according to the actual situation and is not limited here.
[0080] It should be understood that based on the above description, the second service payment data can be the service payment data related to the payment service generated during the second time period to the third time period, and the second service payment data can include the service payment data related to the payment service generated by the associated object associated with the first service object during the second time period to the third time period. Among them, the associated object can be the object involved in the processing of the payment service. For example, when the payment service is a service where an initiating object directly transfers funds to a receiving service object, the first service object is the service object initiating the transfer service (i.e., the initiating object), and the service object associated with the first service object is the receiving object of the transferred assets. For example, in a service where service object A (i.e., the first service object) directly transfers funds to service object B, the associated object associated with service object A is service object B. Another example is when the payment service is a service where an initiating object first transfers funds to a merchant (i.e., the transfer object) and then the merchant transfers funds to the receiving object. The first service object is the service object initiating the transfer service (i.e., the initiating object), and the service objects associated with the first service object can include the merchant and the receiving object. For example, in a service where service object A (i.e., the first service object) first transfers funds to merchant D and then merchant D transfers funds to service object B, the associated objects associated with service object A are service object B and merchant D.
[0081] Here, the method of service payment data for data rebrushing is described in combination with the drawings. Please refer to Figure 7 , Figure 7 is a schematic diagram of the effect of service payment data for data rebrushing provided by an embodiment of the present application. As Figure 7As shown, the sliding processing time window corresponding to the Dth day refers to the sliding processing time window 700a. In the sliding processing time window 700a, it may include the (T + 1)th day, the (T + 2)th day, the (T + 4)th day, and the (T + 5)th day, with each day being a time period. Correspondingly, the Dth day (the first time period) is the day before the (T + 1)th day (i.e., the second time period). At the (T + 1)th day, data scheduling processing can be performed on the business payment data within the Dth day (corresponding to the above-mentioned first business payment data); at the (T + 2)th day, data rebrushing processing can be performed on the business payment data within the Dth day and the (D + 1)th day, and the business payment data of the (D + 1)th day is the above-mentioned second business payment data; at the (T + 2)th day, data rebrushing processing can be performed on the business payment data within the Dth day, the (D + 1)th day, and the (D + 2)th day, and the business payment data within the (D + 1)th day and the (D + 2)th day is the above-mentioned second business payment data; at the (T + 3)th day, data rebrushing processing can be performed on the business payment data within the Dth day, the (D + 1)th day, the (D + 2)th day, and the (D + 3)th day, and the business payment data within the (D + 1)th day, the (D + 2)th day, and the (D + 3)th day is the above-mentioned second business payment data.
[0082] Among them, data rebrushing processing is performed on the first business payment data and the second business payment data based on the data rebrushing link to obtain the second business status data. It can be understood that based on the data calculation nodes on the data rebrushing link, the business status of the payment business initiated by the first business object is re-determined for the first business payment data and the second business payment data. Among them, the business payment data to be rebrushed can be determined based on the first business payment data and the second business payment data, and then data rebrushing processing is performed on the business payment data to be rebrushed based on the data rebrushing link to obtain the second business status data. The business payment data to be rebrushed can be the data used for data rebrushing, and the business payment data to be rebrushed can be obtained by summarizing the first business payment data and the second business payment data. Optionally, when determining the business payment data to be rebrushed based on the first business payment data and the second business payment data, the first business payment data and the second business payment data can be summarized, and data deduplication, cleaning, etc. are performed on the summarized data to obtain the business payment data to be rebrushed.
[0083] Specifically, the first service status data includes the first status data for the target payment service; the first status data includes the service entity information of the target payment service and N service association information corresponding to the service entity information; N is a positive integer; the first status data is determined based on the service payment data for the target payment service in the first service payment data; then, based on the data rebrushing link, data rebrushing processing is performed on the first service payment data and the second service payment data to obtain the second service status data, which may include the following steps: determining the service payment data to be rebrushed based on the first service payment data and the second service payment data; performing data rebrushing processing based on the service payment data to be rebrushed to obtain M service association information associated with the service entity information; M is a positive integer greater than or equal to N; determining the second status data associated with the target payment service based on the service entity information and M service association information, and determining the second service status data based on the second status data.
[0084] Among them, the target payment service may be any payment service initiated by the first service object, and the first status data may be the service status data used to represent that the service status of the target payment service is the first service processing status. It should be understood that the first status data is determined based on the service payment data for the target payment service in the first service payment data. The relevant description for determining the first status data may refer to the relevant description in the above first service status data and will not be elaborated here. The introduction of the service entity information and service association information for the target payment service may refer to the above relevant description and will not be elaborated here. The processing process for determining the service payment data to be rebrushed may refer to the above relevant description and will not be elaborated here.
[0085] Among them, the second status data may be the service status data used to represent that the service status of the target payment service is the second service processing status. It should be understood that in the second status data of the target payment service determined based on the service payment data to be rebrushed, the service entity information and M service association information may be included. Among them, M is a positive integer greater than or equal to N. In other words, M may be the same as N or different from N (i.e., greater than N). It can be understood that the service entity information in the second status data is the same as the service entity information in the first status data. In other words, in the second service status data, the service entity information of any payment service is the same as the service entity information of the same payment service in the first service status data. That is to say, when rebrushing the service status of the payment service initiated within the first time period, the service entity information is still determined based on the service payment data within the first time period, that is, the service entity information remains unchanged during data rebrushing. However, the service association information is determined based on the service payment data of multiple time periods, and the service association information may change.
[0086] For example, in some transfer operations where the initiating entity first transfers funds to a receiving merchant (i.e., the intermediate entity), and then the merchant transfers the funds to the ultimate receiving entity. If the initiating entity initiates the transfer operation on the Tth day (i.e., the first time period) and completes the transfer to the intermediate entity, the first business payment data may include information about the initiation of the transfer operation and information about the merchant's receipt of the transfer. Consequently, the first status data of this payment service may include the business entity information and a business association information, and the business association information includes information about the intermediate entity (such as the merchant's receipt time, merchant identifier, etc.). If the merchant completes the transfer to the ultimate receiving entity on the (T + 1)th day (the second time period), the second business payment data may include information about the ultimate receiving entity's receipt of the transfer (such as the receipt time, the ultimate receiving entity's identifier, etc.). Furthermore, the second status data of this payment service may include the business entity information and a business association information. However, this business association information may include not only the aforementioned information about the intermediate entity (such as the merchant's receipt time, merchant identifier, etc.) but also information about the ultimate receiving entity. It should be understood that after the data of the business status of this transfer operation is refreshed, the number of business association information remains the same (i.e., M = N), but the content of the business association information is updated.
[0087] Another example is in some red - envelope services that can be claimed by multiple people. If the initiating entity initiates the red - envelope service on the Tth day (i.e., the first time period) and entity B claims the red - envelope, the first business payment data may include information about the initiation of this transfer service and information about entity B's claiming of the red - envelope. Consequently, the first status data of this payment service may include the business entity information and a business association information for indicating entity B's claiming of the red - envelope. This business association information may include entity B's identifier, claiming time, claimed amount, etc. If entity C claims the red - envelope on the (T + 1)th day (the second time period), the second business payment data may include information about entity C's claiming of the red - envelope. Furthermore, the second status data of this payment service may include the business entity information, the business association information for indicating entity B's claiming of the red - envelope, and the business association information for indicating entity C's claiming of the red - envelope. It should be understood that after the data of the business status of this red - envelope service is refreshed, the number of business association information increases (i.e., M > N).
[0088] It should be understood that determining the second business status data based on the second status data means that each payment service can be determined as a target payment service to determine the second status data corresponding to each payment service, and the set of the second status data corresponding to each payment service is determined as the second business status data.
[0089] S104. When updating the first service status data in the service payment database based on the second service status data, update the service status of the payment service from the first service processing status to the second service processing status.
[0090] Among them, it should be understood that after the second service status data is determined, the first service status data can be updated based on the second service status data. Specifically, in the service payment database, the first service status data of the corresponding payment service can be rewritten by the second service status data, that is, replaced. For example, in the service database, the first service status data of the target payment service is K1, and the second service status data is K2, then the service status data K1 in the service database can be replaced with the service status data K2. Further, when the service status data in the service payment database is updated, that is, the service status of the payment service is updated, and then subsequent service objects can query the updated service status of the payment service from the service payment database.
[0091] Please refer to Figure 8 , Figure 8 which is a schematic flowchart of a data processing method provided by an embodiment of the present application. This method can be executed by a computer device, such as the server 200a described above. Figure 1 This method can at least include the following steps S201 - step S207.
[0092] S201. When the data processing time reaches the data scheduling time point within the second time period, perform data scheduling processing on the first service payment data generated by the first service object within the first time period through the data calculation node indicated by the service payment system, obtain the first service status data, and store the first service status data in the service payment database; the first time period is the previous time period of the second time period; the second time period belongs to the first sliding processing time window corresponding to the first time period; the first service status data is used to represent that the service status of the payment service initiated by the first service object is the first service processing status.
[0093] It can be understood that, based on the above description, when performing data scheduling and processing, the aggregation processing can be first carried out based on the data aggregation node to obtain the data stored in the data warehouse, and then the data import can be carried out based on the data import node to obtain the data stored in the business payment database. For example, the data warehouse can be the TDW data warehouse, and the business payment data can be the HBase database. As a data warehouse, TDW can store historical data, and there will be an import task every day (i.e., the task processed through the import node) to import the data in TDW into HBase. Generally speaking, the data storage formats of the data warehouse and the business payment database are different. Therefore, when importing the data in the data warehouse into the business payment database through the data import node, format conversion is required. For example, HBase mainly meets the requirements with higher real-time performance. In the enterprise's regular task of importing from TDW to HBase, generally, the column names in TDW correspond to the column names in HBase. In the payment business, such as the red envelope business, if the payment business penetrates to the actual recipient (i.e., the red envelope is received by the business object), there will be a one-to-many situation. In this case, the business entity information is the same, and there will be multiple different records for the actual counterparty (i.e., the receiving object of the business).
[0094] For example, please refer to Figure 9 , Figure 9 which is a schematic diagram of the effect of a kind of business payment data provided by an embodiment of the present application. As Figure 9 shown, in a payment business where multiple people can receive red envelopes, if business object A initiates this business, a business transaction information can be generated (such as shown by 901 in Figure 9 ). When other business objects receive red envelopes, corresponding records can be determined. For example, it can include a record determined based on the business entity information (such as shown by 902 in Figure 9 , such as information like the red envelope initiation time, the red envelope initiation object, the red envelope amount, etc.) and the corresponding business association information 1 (such as shown by 903 in Figure 9 ). The business association information 1 can include the receiving information of a business object 1 (such as information like the received amount, time, etc.); it can also include a record determined based on the business entity information (such as shown by 902 in Figure 9 ) and the corresponding business association information 2 (such as shown by 904 in Figure 9 ). The business association information 2 can include the receiving information of another business object 2 (such as information like the received amount, time, etc.); in addition, it can also include a record determined based on the business entity information (such as shown by 902 in Figure 9 ) and the corresponding business association information 3 (such as shown by 905 in Figure 9 ). The business association information 5 can include the receiving information of another business object 3 (such as information like the received amount, time, etc.).
[0095] It is understandable that in some data warehouses (such as TDW tables), the primary key is the unique identifier of the original transaction order primary key plus the transaction counterparty information after penetration (such as the receiving object of the business). However, if the data format remains unchanged when importing the business status data in TDW (a data warehouse) into the HBase (a database, such as the above-mentioned business payment database) database, the following problems will occur: ① For the one-to-many scenario (such as in a red envelope receiving business, the red envelopes initiated by one business object can be received by multiple objects), the corresponding business entity information is the same, and there are multiple different transaction counterparty information (i.e., transaction association information). After importing into the business payment database (such as HBase), there is redundant storage of the business entity; ② Because the table data in the data warehouse will be emptied and then overwritten when performing data back brushing tasks, for the one-to-many scenario and when the transaction counterparty (i.e., the associated object of the payment business) spans multiple days, there is a situation where the real counterparty (i.e., the associated object of the payment business, such as the receiving object of the red envelope) is not associated during data scheduling. After data back brushing, the real transaction counterparty (i.e., the real associated object, such as the receiving object of the business) is obtained through association. In this way, there may be dirty data when importing into HBase, which will make it difficult to update and import the data in the database through back brushing. To address the above problems, we performed a format conversion when executing the task of importing the business status data in the data warehouse into the business payment database. For example, the business entity information in TDW (data warehouse) and HBase (database) keeps the column names consistent, the Hbase Rowkey design is kept the same as the primary key in the data warehouse (i.e., the key of the main body field information in the business entity information), the transaction counterparty information (i.e., business association information) is identified by the column prefix. Through the opponent column prefix identification and column name conversion, taking advantage of the characteristic that HBase columns can be dynamically linearly extended, the transaction counterparty (associated object) is sparsely stored in the HBase opponent column (the column corresponding to the associated object). In this way, the redundant storage of the transaction entity information in the one-to-many scenario is reduced, and in the case of task back brushing, even if the business spans multiple days for receiving, there will be no dirty data in the primary key, and the back brushing update of the data will be reflected in the dynamic update of the HBase opponent column (i.e., the key-value pair information in the business association information).
[0096] Specifically, the data format of the first business status data is the first data format; the data calculation nodes include a data summary node and a data import node. Then, through the data calculation nodes indicated by the business payment system, data scheduling processing is performed on the first business payment data generated by the first business object within the first time period to obtain the first business status data, which may include the following steps: performing data summary processing on the first business payment data through the data summary node to obtain the initial business status data, and storing the initial business status data in the data warehouse; the data format of the initial business status data is the second data format; the second data format is different from the first data format; performing data import processing on the initial business status data in the data warehouse through the data import node to obtain the first business status data.
[0097] Among them, the data summary node can refer to the relevant description of the above-mentioned summary node, which will not be elaborated here. The data import node can refer to the relevant description of the above-mentioned import node, which will not be elaborated here.
[0098] It can be understood that the data format of the first business status data is the first data format. It can be understood that the specific storage method of the data format of the first business status data can refer to the relevant description above, which will not be elaborated here.
[0099] It should be understood that the initial business status data can be the data obtained by performing data summary processing on the first business payment data through the data summary node, and the data format of the initial business status data is the second data format. In the initial business status data, it may include the initial business entity information and N initial business association information corresponding to the initial business entity information. It should be understood that the initial business entity information can be the business entity information in the second data format, and the initial business association information can be the business association information in the second data format. It should be understood that in the initial business status data in the second data format, for different business association information of the same business entity information, they are all associated and stored with the business entity information. That is to say, if the number of business association information is N, the number of occurrences of the business entity information also includes at least N times. It can be understood that the second data format is different from the first data format.
[0100] Specifically, the initial business state data includes the initial state data for the target payment service; the data format of the initial state data is the second data format; the initial state data includes the initial business entity information of the target payment service and N initial business association information corresponding to the initial business entity information; N is a positive integer; in the initial state data, each of the N initial business association information is stored in a one-to-one association with the initial business entity information, and the storage times of the initial business entity information is N; each of the initial business association information includes at least one association field information; the initial business entity information includes at least one entity field information; then, through the data import node, the initial business state data in the data warehouse is subjected to data import processing to obtain the first business state data, which may include the following steps: determining a main key value information for each entity field information included in the initial business entity information respectively to obtain at least one main key value information; one entity field information corresponds to one main key value information; at least one main key value information is used to indicate the business entity information of the target payment service; determining at least one association field information in each initial business association information as an association data value information, and determining an association key value information corresponding to each initial business association information based on the association data value information corresponding to each initial business association information; the association data value information of one initial business association information corresponds to one association key value information; the association key value information corresponding to each initial business association information is used to indicate the business association information of the target payment service; through the data import node, data import processing is performed on at least one main key value information in the data warehouse and the association key value information corresponding to each initial business association information to obtain the first state data corresponding to the target payment service; the data format of the first state data is the first data format; determining the first business state data based on the first state data.
[0101] It can be understood that the introduction of the association field information and the entity field information can refer to the above relevant description, and will not be elaborated here. It can be understood that the introduction of the main key value information and the association key value information can refer to the above relevant description, and will not be elaborated here.
[0102] Among them, the initial state data can be the state data when the target payment service is stored in the data warehouse. It can be understood that the initial state data may include initial business entity information and the corresponding N initial business association information. It should be understood that the initial business entity information can be the business entity information of the target payment service included in the initial state data, and the initial business association information can be the business association information of the target payment service included in the initial state data. It can be understood that the initial business entity information is consistent with the business entity information stored in the above-mentioned business payment database in terms of content, but different in data format; the initial business association information is consistent with the business association information stored in the above-mentioned business payment database in terms of content, but different in data format. For the relevant introduction of the content of the initial business association information and the initial business entity information, reference can be made to the relevant descriptions of the above-mentioned business association information and business entity information, which will not be elaborated here. Among them, the associated data value information can be the information used to determine the value in the associated key value information. The associated data value information can be a string obtained by combining N associated field information.
[0103] The process of data import is described below with reference to the drawings. Please refer to Figure 10 , Figure 10 which is a schematic diagram showing the effect of a data import process provided by an embodiment of the present application. As Figure 10As shown, in the data warehouse 1001b, for a business entity, subject field information and associated field information can be recorded. For each piece of associated field information, the same subject field information can be recorded. For example, the field value corresponding to field 1 is "123456", the field value corresponding to field 2 is "xxxx", and the field value corresponding to field 3 is "45678", thus storing the same subject field information multiple times. Any piece of associated field information can include information about fields such as field 4, field 5, field 6, and field 7. Further, when importing the data in the data warehouse 1001b into the business payment database 1002b, the field information of each field in the subject field information can be determined as a subject key-value information. For example, in the business entity information 1003b, it can include key-value pairs with the key being field 1 and the value being 123456, the key being field 2 and the value being xxxx, and the key being field 3 and the value being 45678. In the business association information 1004b, it can include key-value pairs determined for each piece of business association information. For example, it can include the key-value pair determined based on association information 1, with the key being a string indicating association information 1 and the value being the set of association information 1 (i.e., set 1, which is the above-mentioned associated key-value information). Among them, set 1 can include V11, V12, V13, and V14. For example, this set 1 can be the string {"field 1": "V11", "field 2": "V12", "field 3": "V13", "field 4": "V14"}; it can also include the key-value pair determined based on association information 2, with the key being a string indicating association information 2 and the value being the set of association information 2 (i.e., set 2, which is the above-mentioned associated key-value information). Among them, set 2 can include V21, V22, V23, and V24. For example, this set 2 can be the string {"field 1": "V21", "field 2": "V22", "field 3": "V23", "field 4": "V24"}; it can also include the key-value pair determined based on association information 3, with the key being a string indicating association information 3 and the value being the set of association information 3 (i.e., set 3, which is the above-mentioned associated key-value information). Among them, set 1 can include V31, V32, V33, and V34. For example, this set 3 can be the string {"field 1": "V31", "field 2": "V32", "field 3": "V33", "field 4": "V34"}.
[0104] S202. When reaching the data rebrushing time point of the data within the third time period in the first sliding processing time window at the data processing time, based on the data calculation nodes associated with the payment business, determine the data backhaul link for changing the first business processing state; the third time period is any time period in the first sliding processing time window that is later than the second time period.
[0105] It can be understood that, since the computing resources are limited, in order not to affect the regular data scheduling, it is usually the case that the data reshing is selected at a time when the computing resources of the computer device are relatively idle, thereby reasonably utilizing the computing resources, avoiding overloading of the computer device, and not affecting the operation of the incremental instance (i.e., regular data scheduling processing).
[0106] Specifically, the embodiment of the present application may further include the following steps: obtaining resource occupancy information of a service processing device associated with a data computing node, and determining a data reshing time point within a third time period based on the resource occupancy information.
[0107] Among them, the resource occupancy information can be used to indicate the occupancy situation of the computing resources of the computer device. The resource occupancy information can be expressed as indicating the proportion of the used computing occupancy in all computing resources in the computer device, or can also be expressed as the size of the remaining computing resources (i.e., the unoccupied computing resources), etc., which is not limited here.
[0108] It can be understood that determining the data reshing time point within the third time period based on the resource occupancy information can be to determine the data processing time when the resource occupancy information meets certain preset conditions (also called reshing conditions) as the data reshing time point. The reshing conditions can be the conditions for determining the data reshing time point for data reshing. The reshing conditions can be that the occupancy ratio indicated by the resource occupancy information (i.e., the proportion of the used computing occupancy in all computing resources in the computer device) is less than a certain threshold, or the occupancy ratio indicated by the resource information is less than a certain threshold for a certain duration; or, the reshing conditions can be that the remaining computing resources (i.e., the unoccupied computing resources) indicated by the resource occupancy information are greater than a certain threshold, or, or the occupancy ratio indicated by the resource information is greater than a certain threshold for a certain duration, etc., which is not limited here.
[0109] Optionally, the data reshing time point can be a preset time point within the time period, and the preset time point can be a time point determined according to the historical resource occupancy situation. The historical resource occupancy situation can be the resource occupancy situation within a time period before the first time period. The time point with the least average resource occupancy in the historical resource occupancy situation can be determined as the data reshing time point. For example, the data reshing time point can be 12:00 noon within a day.
[0110] For example, please refer to Figure 11 , Figure 11 is a schematic diagram of the effect of the resource occupancy situation of a data processing provided by the embodiment of the present application. As Figure 11As shown, the horizontal axis represents time, and the vertical axis is used to characterize the memory usage trend. The memory usage trend can be represented by the amount of memory used or by the usage rate, and there is no limitation here. Among them, the data scheduling operation peak 1101a can indicate that this period of time is the peak of memory (i.e., computing resources) usage due to data scheduling processing. The data reshing operation peak 1102a can be used to indicate that this period of time is the peak of memory (i.e., computing resources) usage due to data reshing processing. It can be seen that within a time period (such as within a day), the data scheduling operation peak 1101a is earlier than the data reshing operation peak 1102a, thereby balancing the resource occupancy caused by data reshing processing and data scheduling processing, and avoiding the data reshing processing from occupying the computing resources required for data scheduling processing and affecting the normal execution of data scheduling processing.
[0111] It can be understood that when determining the data reshing link, the data reshing link can be determined based on the above-mentioned computing node topology diagram. For example, a computing node topology diagram associated with the business payment system can be obtained; in the computing node topology diagram, a computing node matching the data computing node is determined, and the data reshing link is determined based on the computing node matching the data computing node.
[0112] Specifically, determining the data reshing link for changing the first service processing state based on the data computing node associated with the payment service may include the following steps: obtaining a computing node topology diagram associated with the business payment system; the computing node topology diagram includes computing nodes and edges; a computing node is used to indicate the computing logic for data processing; an edge is determined based on the computing order between the two connected computing nodes; in the computing node topology diagram, a computing node matching the data computing node is determined, and based on the computing node matching the data computing node and the edge between the computing nodes matching the data computing node, the data reshing link for changing the first service processing state is determined.
[0113] Among them, the introduction of the computing node topology diagram can refer to the above relevant description and will not be elaborated here. It should be understood that in the business payment system, the data computing node of the payment service initiated by the first business object may exist in the corresponding computing node topology diagram, and then a computing node matching the above data computing node can be determined in the computing node topology diagram, so that based on the computing node matching the data computing node and the edge between the computing nodes matching the data computing node, the data reshing link for changing the first service processing state can be determined.
[0114] For example, please refer to Figure 12 , Figure 12 is a schematic diagram of the effect of a data reshing link provided by an embodiment of the present application. AsFigure 12 As shown, on this data rebrushing link, it may include a starting node m1 (such as the starting node 1201a shown in Figure 12 ), a scenario node (such as multiple nodes a1, a2, a3, a4, etc. included in the scenario node 1202a area shown in Figure 12 ), a verification node 1203a (such as multiple nodes b1, b2,....... b6, etc. included in the verification node 1203a area shown in Figure 12 ), a summarization node (such as node c1, such as the summarization node 1204a shown in Figure 12 ), an import node (such as nodes d1, d2,....... d4, etc. within the area indicated by the import node 1205a shown in Figure 12 ), where nodes d1, d3, and d4 are the end nodes on this data rebrushing link. On this data rebrushing link, it may also include an elimination node 1207a (which can be used to eliminate dirty data, duplicate data, etc.) and other data calculation nodes such as the other node 1206a, which are not limited here. It should be understood that on this data rebrushing link, the calculation order between each data calculation node is indicated by the connection direction of the edges. For example, if the connection direction between the starting node 1201a and node a1 is from the starting node 1201a to node a1, then the starting node 1201a is calculated first and then node a1.
[0115] S203. Perform data rebrushing processing on the first service payment data and the second service payment data based on the data rebrushing link to obtain second service status data; the second service payment data includes service payment data related to the payment service generated by the associated object associated with the first service object during the second time period to the third time period; the second service status data is used to represent that the service status of the payment service is the second service processing status.
[0116] Optionally, based on the above description, the number of the third time periods can be single, that is, the third time period is the next time period of the second time period. Then, the second service payment data determined during data rebrushing can be the service payment data during the second time period.
[0117] Specifically, the third time period is the next time period of the second time period; then, performing data rebrushing processing on the first service payment data and the second service payment data based on the data rebrushing link to obtain second service status data may include the following steps: determining the service payment data related to the payment service generated by the associated object during the second time period as the second service payment data; performing data rebrushing processing on the first service payment data and the second service payment data based on the data rebrushing link to obtain second service status data.
[0118] Further, it can be understood that within any time period, in addition to being able to re-brush the data of payment services initiated within the time period corresponding to the time window to which it belongs, regular data scheduling processing can also be performed on the previous time period. The process of data scheduling data can refer to the relevant description of data scheduling processing for the first business payment data above, which will not be elaborated here. In order for the re-brush task (i.e., the task of performing data re-brushing) not to affect the execution of the incremental task (i.e., the task of performing data scheduling processing), in the embodiments of the present application, an incremental task execution dependency can be set (i.e., the incremental task is preferentially executed, and the re-brush task is performed after the incremental task is completed), and the re-brush start time point (i.e., the data re-brush time point) is usually set at a time when the computing resources of the computing cluster (i.e., the computer device on which the present application runs) are relatively idle (such as at noon in a day), so as to minimize the impact on the operation of the incremental instance (i.e., the incremental task) while balancing the utilization rate of the computing resources of the computing cluster.
[0119] Specifically, the second business payment data includes the business payment data generated by the second business object within the second time period; the third time period belongs to the second sliding processing time window corresponding to the second time period, and the data scheduling time point within the third time period is earlier than the data re-brush time point within the third time period; the third time period is the next time period of the second time period; then the embodiments of the present application may further include the following steps: when the data processing time reaches the data scheduling time point within the third time period, based on the data calculation node associated with the payment service, through the data calculation node indicated by the business payment system, perform data scheduling processing on the second business payment data to obtain the third business status data, and store the third business status data in the business payment database; the third business status data is used to represent that the business status of the payment service initiated by the second business object is the third business processing status.
[0120] Among them, the second business object may be a business object that initiates a payment service within the second time period. That is to say, the second business payment data includes the business payment data associated with the payment service generated by the second business object within the second time period. The number of second business objects may be one or more. It should be understood that the second business object and the first business object may be the same or different, which is not limited here. For example, if a business object initiates a payment service within the first time period and also initiates a payment service within the second time period, then there may be the same business object among the first business object and the second business object.
[0121] It should be understood that when the number of the third time periods is single, the third time period belongs to the second sliding processing time window corresponding to the second time period. The second sliding processing time window may be the sliding processing window corresponding to the second time period. The method for determining the second sliding processing time window may refer to the relevant description of determining the first sliding processing time window above, which will not be elaborated here.
[0122] The sliding process of the sliding processing time window is described herein with reference to the drawings. Please refer to Figure 13 , Figure 13 which is a schematic diagram of the sliding of a sliding processing time window provided by an embodiment of the present application. As Figure 13 shown, the size of the sliding processing time window determined for the payment service is 3 time periods. The sliding processing time window determined for the 1st time period is the sliding processing time window 1, which may include the 2nd time period, the 3rd time period, and the 4th time period. Further, when the data processing time reaches the 2nd time period, the sliding processing time window determined for the 2nd time period is the sliding processing time window 2, which may include the 3rd time period, the 4th time period, and the 5th time period. Similarly, when the data processing time reaches the 3rd time period, the sliding processing time window determined for the 3rd time period is the sliding processing time window 3, which may include the 4th time period, the 5th time period, and the 6th time period. And so on. When the data processing time reaches each time period, a corresponding sliding processing time window can be determined to continuously update the service status of the payment service initiated within the corresponding data period, ensuring the integrity of the service data of the payment service finally obtained, that is, the service data (i.e., service status data) of the payment service can cover the entire processing process of the payment service.
[0123] It should be understood that based on the above description, it can be known that the business payment system may generate certain business payment data in each time period, that is, the business payment data of the business payment system is divided into each time period, that is, a time period may correspond to a partition range of business payment data (which may also be referred to as a data partition range). It can be understood that when determining the service status of the payment service initiated in a time period, the business payment data relied on may be the time periods included when the corresponding sliding processing time window moves forward by one time period.
[0124] For example, please refer to Figure 14 , Figure 14 which is a schematic diagram of the determination of business payment data provided by an embodiment of the present application. As Figure 14As shown, the sliding processing time window 1401a can be the sliding processing time window corresponding to the first time period. This sliding processing time window 1401a can include the second time period, the third time period, and the fourth time period. Then, when determining the service status of the payment service initiated within the first time period based on the sliding processing time window 1401a, the service payment data relied on can be the first time period, the second time period, and the third time period, such as Figure 10 the data partition range corresponding to the sliding processing time window 1401a in
[0125] It should be understood that within the third time period, data scheduling processing can be performed on the service payment data within the second time period. Considering the above purpose of not affecting the execution of the incremental task (i.e., the task of data scheduling processing) in order to implement the backbrush task (i.e., the task of data backbrushing), the data scheduling time point within the third time period can be earlier than the data rebrushing time point within the third time period. Additionally, it can also be set that within the third time period, after the data scheduling processing of the service payment data within the second time period is completed and the data processing time reaches the data rebrushing time point within the third time period, data rebrushing is then performed.
[0126] It should be understood that the third service status data is used to represent that the service status of the payment service initiated by the second service object is the third service processing status, that is, the service status of the payment service initiated by the second service object within the second time period. It can be understood that the processing process for determining the third service status data can refer to the above processing process for determining the second service status data, and details are not elaborated here.
[0127] Optionally, based on the above description, the number of third time periods can be multiple, that is, the third time period can include multiple sub-time periods. Then, the second service payment data determined during data rebrushing can be the service payment data within the second time period, or the second service payment data can include the service payment data within the second time period and the service payment data within one or more sub-time periods within the third time period.
[0128] Optionally, the third time period includes a first sub-time period and a second sub-time period; the first sub-time period is the next time period of the second time period, and the second sub-time period is the next time period of the first sub-time period; the data reflushing time point within the third time period includes the first data reflushing time point within the first sub-time period; the data backflushing link is determined when the data processing time reaches the first data reflushing time point; then, based on the data backflushing link, performing data reflushing processing on the first service payment data and the second service payment data to obtain the second service status data may include the following steps: determining the service payment data associated with the payment service generated within the second time period as the second service payment data; performing data reflushing processing on the first service payment data and the second service payment data based on the data backflushing link to obtain the first sub-service status data, and determining the second service status data based on the first sub-service status data.
[0129] Wherein, the first sub-time period and the second sub-time period can be any time periods included in the third time period. Wherein, the first sub-time period is the next time period of the second time period, and the second sub-time period is the next time period of the first sub-time period. It should be understood that a data reflushing time point can be included within both the first sub-time period and the second sub-time period. The data reflushing time point within the first sub-time period can be referred to as the first data reflushing time point. In other words, the first data reflushing time point is the data reflushing time point within the first sub-time period; the data reflushing time point within the second sub-time period can be referred to as the second data reflushing time point. In other words, the second data reflushing time point is the data reflushing time point within the second sub-time period. It can be understood that the data backflushing link can be determined when the first data reflushing time point is reached. Specifically, when the data processing time reaches the first data reflushing time point, based on the data calculation nodes associated with the payment service, the data backflushing link for changing the first service processing status is determined. It can be understood that the first sub-time period and the second sub-time period here are only examples, and the third time period can also include a greater number of sub-time periods, such as a third sub-time period, a fourth sub-time period, etc., which are not limited here.
[0130] Wherein, it should be understood that when the data processing time reaches the first data reflushing time point, the second service payment data can be the service payment data associated with the payment service generated within the second time period, that is, the service payment data generated within the second time period. The service payment data generated within the second time period can include the service payment data generated by the associated object of the first service object.
[0131] Among them, the first sub-business status data is used to represent that the business status of the payment business initiated by the first business object is the first sub-business processing status, and the first sub-business status is also the business status of the payment business determined within the first sub-time period. In other words, the first sub-business status data can be used to represent the business status of the payment business determined within the first sub-time period, that is, the business status of the payment business at the end of the second time period. Furthermore, the second business status data can be determined based on the first sub-business status data, that is, the first sub-business status data is determined as the second business status data. The method for determining the first sub-business status data based on the first business payment data and the second business payment data can refer to the relevant description of determining the second business status data based on the first business payment data and the second business payment data above, which will not be elaborated here.
[0132] Optionally, the third time period includes a first sub-time period and a second sub-time period; the first sub-time period is the next time period of the second time period, and the second sub-time period is the next time period of the first sub-time period; the data rebrushing time points within the third time period include a first data rebrushing time point within the first sub-time period and a second data rebrushing time point within the second sub-time period; the second business status data includes the first sub-business status data; the first sub-business status data is used to represent that the business status of the payment business initiated by the first business object is the first sub-business processing status; the first sub-business status data is determined based on the first business payment data and the business payment data generated within the second time period when the data processing time reaches the first data rebrushing time point; the data rebrushing link is determined when the data processing time reaches the second data rebrushing time point; then, based on the data rebrushing link, performing data rebrushing processing on the first business payment data and the second business payment data to obtain the second business status data may include the following steps: determining the business payment data associated with the payment business generated within the second time period and the business payment data associated with the payment business generated within the first sub-time period as the second business payment data; performing data rebrushing processing on the first business payment data and the second business payment data based on the data rebrushing link to obtain the second sub-business status data, and determining the second business status data based on the first sub-business data and the second sub-business format data.
[0133] Among them, the determination process of the first sub-business status data can refer to the above description, which will not be elaborated here. That is to say, the first sub-business status data is determined based on the first business payment data and the business payment data generated within the second time period when the data processing time reaches the first data rebrushing time point. It can be understood that when the data processing time point reaches the second data rebrushing time point, the data rebrushing link can also be determined for data rebrushing within the second time period.
[0134] It should be understood that when the data processing time reaches the second data rebrushing time point, the second business payment data can be the business payment data associated with the payment business generated within the second time period, as well as the business payment data associated with the payment business generated within the first sub-time period, that is, the business payment data generated within the second time period and the business payment data generated within the first sub-time period.
[0135] Among them, the second sub-business status data is used to characterize that the business status of the payment business initiated by the first business object is the second sub-business processing status, and the second sub-business status is also the business status of the payment business determined within the second sub-time period. In other words, the second sub-business status data can be used to characterize the business status of the payment business determined within the second sub-time period, that is, to characterize the business status of the payment business at the end of the first sub-time period. Furthermore, the second business status data can be determined based on the first sub-business status data and the second sub-business status, that is, the first sub-business status data is determined as the second business status data. The method for determining the second sub-business status data based on the first business payment data and the second business payment data can refer to the relevant description of determining the second business status data based on the first business payment data and the second business payment data above, and will not be elaborated here.
[0136] It should be understood that after the data processing time reaches the second data rebrushing time point, the determined second business status data can be used to update the data after the data processing time reaches the first data rebrushing time point, so as to obtain a new second business status data, and the second business status data stored in the business payment database is updated through the new second business status data.
[0137] It can be understood that in the third time period, there can also be a greater number of sub-time periods, and the specific processing processes can all refer to the relevant description of the data processing process when reaching the data rebrushing time point within the second sub-time period, and will not be elaborated here.
[0138] Furthermore, the second service payment data includes the service payment data generated by the third service object within the first sub-time period. Then, the embodiments of the present application may further include: when the data processing time reaches the data scheduling time point within the second sub-time period, based on the data calculation node associated with the payment service, through the data calculation node indicated by the service payment system, perform data scheduling processing on the service payment data generated within the first sub-time period to obtain the fourth service status data, and store the fourth service status data in the service payment database; the fourth service status data is used to represent that the service status of the payment service initiated by the third service object is the fourth service processing status; the second sub-time period belongs to the third sliding processing time window corresponding to the first sub-time period, and the data scheduling time point within the second sub-time period is earlier than the data rebrushing time point within the second sub-time period.
[0139] Among them, the fourth service status data is used to represent that the service status of the payment service initiated by the third service object is the fourth service processing status (which can also be called the fourth service status). This fourth service processing status is the service status of the payment service initiated within the first sub-time period determined within the second sub-time period. Among them, the process of performing data scheduling processing can refer to the relevant description of determining the third service processing status above, and will not be elaborated here.
[0140] The third sliding processing time window is the sliding processing time window corresponding to the first sub-time period. The determination method of the third sliding processing time window can refer to the relevant description of the first sliding processing time window above, and will not be elaborated here.
[0141] S204. When updating the first service status data in the service payment database based on the second service status data, update the service status of the payment service from the first service processing status to the second service processing status.
[0142] Step S204 can refer to the relevant description of step S104 above, and will not be elaborated here.
[0143] S205. When the data processing time reaches the data rebrushing time point within the fourth time period, based on the data calculation node associated with the payment service, determine the data backbrush link for changing the second service processing status; the fourth time period belongs to the jump time window corresponding to the first time period; the jump time window is later than the last time period within the first sliding processing time window.
[0144] It can be understood that in some payment services, the time point of a change in a certain business state of the payment service is usually after a fixed duration after the payment service is initiated. For example, in some enterprise payment services, after the initiator makes a payment, the platform makes a unified payment transfer to the receiving object after a fixed 7 days. If this fixed duration is relatively long, a relatively large sliding processing time window needs to be used to cover the entire processing process of the payment service. However, if the size of the sliding processing time window is blindly increased, a large amount of data that needs to be rebrushed will be added, resulting in data calculation performance issues, and there will be redundant data rebrushing in a certain period in the middle. Therefore, for some payment service scenarios, a method combined with a jumping time window can be used for data rebrushing processing. As shown in the following figure, the jumping time window only selects the partition data of the starting time (i.e., the business payment data within the time period when the payment service is initiated, such as the first business payment data) and the partition data of a fixed jumping time (i.e., the business payment data within a preset fixed time period) for data rebrushing. For situations where the business state needs to be updated after a fixed number of days in some special scenarios, a rolling backbrush combining the jumping time window and the sliding processing time window can be adopted to avoid expanding the sliding processing time window and increasing the scanned time partition range (i.e., increasing the amount of data for data rebrushing processing), thereby improving the calculation performance.
[0145] Among them, the fourth time period can belong to the jumping time window corresponding to the first time period. It can be understood that this jumping time window can be determined according to the change time of the business state of the payment service. For example, if the payment service makes a change in the business state once in the Kth time period after the payment service is initiated, the jumping time window can be determined based on the (K + 1)th time period. It can be understood that one or more time periods can be included within this jumping time window, and the fourth time period can be any time period within the jumping time window. For example, if the payment service makes a change in the business state once within the Kth to (K + 1)th time periods after the payment service is initiated, the change can be made in the Kth time period or within the (K + 1)th time period, then the jumping time window can be determined based on the (K + 1)th time period and the (K + 2)th time period.
[0146] It should be understood that when combining the jumping time window with the sliding processing time window, the jumping time window should be later than the last time period within the sliding processing time window, and the time periods within the jumping time window do not overlap with the time periods within the sliding processing time window.
[0147] It should be understood that the method for determining the data re - brush time point within the fourth time period can refer to the above description and will not be elaborated here. The method for determining the data back - brush link for changing the second service processing status can refer to the relevant description of determining the data back - brush link for changing the first service processing status above and will not be elaborated here. In addition, since the logic for processing the business payment data of the same payment service should be consistent, the determined data back - brush link can be the same each time.
[0148] For example, please refer to Figure 15 , Figure 15 which is a schematic diagram for determining business payment data provided by an embodiment of the present application. As Figure 15 shown, the jump time window 1501a can be the jump time window corresponding to the first time period, that is, this jump time window 1501a is the eighth time period. When determining the service status of the payment service initiated within the first time period based on the jump time window 1501a, the business payment data relied on can include the first time period and seven time periods, such as Figure 15 the data partition range corresponding to the jump time window 1501a in it. In other words, this data partition range can include the first time period and the seventh time period.
[0149] Furthermore, please refer to Figure 16 , Figure 16 which is a schematic diagram for determining business payment data provided by an embodiment of the present application. Figure 16 When combining the sliding time window and the jump time window, it is the business payment data relied on when determining the service status of the payment service initiated within the first time period. Among them, the sliding processing time window for the first time period is the sliding processing time window 1601a, and the corresponding business payment data can be the business payment data within the first time period to the third time period. The jump time window for the first time period is the jump time window 1602a, and the corresponding business payment data can be the first time period and the seventh time period. Therefore, when combining the sliding processing time window 1601a and the jump time window 1602a, for determining the payment service initiated within the first time period, the business payment data relied on refers to that shown in the relied - on business payment data 1603a, that is, it can include the first time period, the second time period, the third time period, and the seventh time period. Among them, the business payment data within the first time period corresponds to the above - mentioned first business payment data, the business payment data within the second time period and the third time period can be the above - mentioned second business payment data, and the business payment data within the seventh time period can be the above - mentioned third business payment data.
[0150] S206. Perform data rebrushing processing on the first service payment data, the second service payment data, and the third service payment data based on the data rebrushing link to obtain the fifth service status data. The third service payment data includes service payment data associated with the payment service generated by an associated object associated with the first service object in the time period before the fourth time period. The fifth service status data is used to represent that the service status of the payment service is the fifth service processing status.
[0151] Among them, the fifth service status data can be used to represent that the service status of the payment service is the fifth service processing status, and this fifth service processing status can be the service status of the payment service determined within the fourth time period. In other words, the fifth service status data can be used to represent the service status of the payment service determined within the fourth time period, that is, to represent the service status of the payment service at the end of the time period before the fourth time period.
[0152] It can be understood that the third service payment data can include service payment data associated with the payment service generated by an associated object associated with the first service object in the time period before the fourth time period. It can be understood that if the number of the fourth time periods is single, the third service payment data can be the service payment data generated in the time period before the fourth time period. If the number of the fourth time periods is multiple, and the fourth time period where the data rebrushing time point reached by the data processing time is the first time period of the jump time window, the third service payment data can be the service payment data generated in the time period before the fourth time period. If the number of the fourth time periods is multiple, and the fourth time period where the data rebrushing time point reached by the data processing time is other time periods except the first time period in the jump time window, the third service payment data can be the service payment data generated in the time period before the fourth time period, and the service payment data generated in the fourth time periods before the fourth time period where the data rebrushing time point is located. It should be understood that within the jump time window, there can be corresponding data rebrushing time points in each time period, and when the data processing time reaches each data rebrushing time point, a data rebrushing (also called data rebrushing processing) can be performed once.
[0153] It should be understood that performing data rebrushing processing on the first service payment data, the second service payment data, and the third service payment data based on the data rebrushing link to obtain the fifth service status data can refer to the relevant description of performing data rebrushing processing on the first service payment data and the second service payment data based on the data rebrushing link to obtain the second service status data, which will not be elaborated here.
[0154] S207. When updating the second service status data in the service payment database based on the fifth service status data, update the service status of the payment service from the second service processing status to the fifth service processing status.
[0155] It should be understood that after determining the fifth service status data, the second service status data can be updated based on the fifth service status data. Specifically, in the service payment database, the second service status data can rewrite, that is, replace, the fifth service status data of the corresponding payment service. For example, in the service database, the second service status data of the target payment service is K2, and the fifth service status data is K3, then the service status data K2 in the service database can be replaced with the service status data K3. Further, when updating the service status data in the service payment database, it also updates the service status of the payment service. Subsequently, the subsequent service object can query the updated service status of the payment service from the service payment database.
[0156] Optionally, in the embodiment of the present application, a page for viewing the processing status of service payment data can also be provided. Please refer to Figure 17 , Figure 17 which is a schematic diagram of the effect of a query page for the processing status of a type of data provided by the embodiment of the present application. As Figure 17 shown, page 170a is a page for viewing the processing status of service payment data. In this page 170a, it can include an area (such as the area shown as 1701a) for selecting the query range of the service payment data to be queried. Among them, the displayed query range is that the data start time (i.e., the generation time of the service payment data, such as the first time period, the second time period, etc.) is from xx-11-11 to xx-11-15. In this page 170a, it can also include a control for indicating confirmation of the query (such as the control shown as 1702a). When the user (also called the query object) performs a trigger operation on this control 1702a (such as clicking on this control 1702a), the processing status of the service payment data with the data start time within this query range is displayed. Optionally, in this page 170a, it can also include an area (such as the area shown as 1703a) for selecting the processing status of the service payment data in which processing status to display. In this area, processing statuses such as all processing statuses, status 1, status 2, etc. can be selected. The processing status here can be ready (i.e., the state of waiting for scheduling or rebrushing), waiting for the parent task (i.e., the state of waiting for the parent task to be processed. For example, some data rebrushing tasks need to wait for the data scheduling task to be processed), running (the state of being in the process of processing), success, failure, etc. The specific status is not limited here and is determined by actual requirements.
[0157] It can be understood that the processing status information of the queried business payment data can be displayed on the page 170a. The processing status information of a business payment data may include the data start time and the data execution time (i.e., the start and end times of data processing). If the processing has not started, the start time and end time included in the data execution time are default values or null values. If the processing has started but not ended, the start time included in the data execution time is the actual start time, and the end time is the default value or null value. If the processing has ended, the start time and end time included in the data execution time are actual time values. Optionally, the processing status information of a business payment data may include the execution duration. If the execution has ended, the execution duration is the time difference between the start time and the end time. If the execution is not completed, the execution duration is the default value or null value.
[0158] It can be understood that based on the above description, in order not to affect the execution of the incremental task, the data scheduling process of the incremental data can be performed first, and then the data backbrush process of the stock data can be performed. For example, referring to Figure 17 As shown, if the incremental data time 1704a is xx-11-14 00:00:00, it means that the data for which data scheduling processing needs to be performed for this record is the business payment data generated within xx-11-14, and the time range of the data scheduling processing (i.e., the data scheduling processing time 1705a) is "xx-11-15 08:22:00" - "xx-11-15 08:56:00". If the stock data time 1706a is 00:00:00 of xx-11-11, it means that the data for which data rebrushing processing needs to be performed for this record is the business payment data generated within xx-11-11, and the time range of the data rebrushing processing (i.e., the data rebrushing processing time 1707a) is "xx-11-15 13:57:00" - "xx-11-15 14:15:00". It can be seen that on xx-11-15, in order not to affect the data scheduling processing of the incremental data (i.e., the business payment data in the previous time period, such as the business payment data within xx-11-14), the time of the data scheduling processing (as shown in the data scheduling processing time 1705a) is earlier than the time of the data rebrushing processing of the stock data (as shown in the data rebrushing processing time 1707a) (i.e., the business payment data in the time period before the previous time period within the sliding processing time window, such as the business payment data on xx-11-11).
[0159] Here, in combination with the illustration, the backbrush effect of the present application is described. Please refer to Figure 18 , Figure 18 is a schematic diagram of a backbrush effect provided by an embodiment of the present application. As Figure 18As shown, the horizontal axis represents time (such as the 07th - 15th day of a month), and the vertical axis is used to represent the missing rate information, which can be used to represent the missing rate of the business status of the payment service (i.e., the associated missing rate). Here, the missing means the failure to associate the business association information of the payment service with the corresponding business entity information. It can be understood that the missing rate information can be expressed as the ratio of the payment services whose business status cannot be fully determined within a time period (i.e., the payment services missing business association information) to all payment services, or it can be expressed as the number of payment services whose business status cannot be fully determined within a time period. There is no limitation here. It can be seen that within the time period of the 12th day, the corresponding missing rate information approaches 0 because the corresponding sliding processing time window is from the 13th day to the 15th day. On the 13th day, data scheduling processing was performed for the 12th day, and on the 14th and 14th days, data rebrushing processing was performed for the 12th day. After the rebrushing processing, the missing rate of the business status of the payment services within the 12th day approaches 0. The reason why the missing rate corresponding to the time period from the 07th to the 11th day approaches 0 is the same. Within the time period of the 13th day, the corresponding missing rate information does not approach 0 because the sliding processing time window corresponding to the 13th day is from the 14th day to the 16th day. On the 13th day, data scheduling processing was performed for the 12th day, and on the 16th day, data rebrushing processing was performed for the 12th day, but at this time, data backbrushing processing for the 16th day was not performed. Therefore, there is still a part of the payment services whose business status is missing. Similarly, it can be known that the sliding processing time window for the 14th day is from the 15th day to the 17th day, but at this time, only data scheduling processing was performed on the 15th day, and no data backbrushing processing was performed, so there is still a part of the payment services whose business status is missing, and the missing rate of the 14th day is larger than that of the 13th day (because data backbrushing was performed for the payment services on the 13th day, but no data backbrushing was performed for the payment services on the 14th day). Thus, it can be seen that through the data rebrushing scheme of the embodiments of the present application, the integrity of the business status of the payment service is greatly improved.
[0160] Please refer to Figure 19 , Figure 19 is a schematic structural diagram of a data processing device provided by an embodiment of the present application. As Figure 19 shown, the data processing device 1 can be a computer program (including program code) running on a computer device. For example, the data processing device 1 is an application software. It can be understood that the data processing device 1 can be used to execute the corresponding steps in the data processing method provided by the embodiments of the present application. As Figure 19 shown, the data processing device 1 can include: a data scheduling module 11, a backbrushing link determination module 12, a data backbrushing module 13, and a data update module 14;
[0161] A data scheduling module 11, which is used to perform data scheduling processing on first business payment data generated by a first business object within a first time period when the data processing time reaches a data scheduling time point within a second time period, through a data calculation node indicated by a business payment system, to obtain first business status data, and store the first business status data in a business payment database; the first time period is the previous time period of the second time period; the second time period belongs to a first sliding processing time window corresponding to the first time period; the first business status data is used to represent that the business status of the payment business initiated by the first business object is a first business processing status;
[0162] A backbrush link determination module 12, which is used to determine a data backbrush link for changing the first business processing status based on a data calculation node associated with the payment business when the data processing time reaches a data rebrushing time point within a third time period in the first sliding processing time window; the third time period is any time period in the first sliding processing time window that is later than the second time period;
[0163] A data backbrush module 13, which is used to perform data rebrushing processing on the first business payment data and second business payment data based on the data backbrush link to obtain second business status data; the second business payment data includes business payment data associated with the payment business generated by an associated object associated with the first business object within the second time period to the third time period; the second business status data is used to represent that the business status of the payment business is a second business processing status;
[0164] A data update module 14, which is used to update the first business status data in the business payment database based on the second business status data, and update the business status of the payment business from the first business processing status to the second business processing status.
[0165] Wherein, the third time period is the next time period of the second time period;
[0166] Wherein, the data backbrush module 13 includes a first rebrushing unit 131;
[0167] The first rebrushing unit 131 is used for:
[0168] Determine the business payment data associated with the payment business generated within the second time period as the second business payment data;
[0169] Perform data rebrushing processing on the first business payment data and the second business payment data based on the data backbrush link to obtain the second business status data.
[0170] Among them, the second service payment data includes the service payment data generated by the second service object within the second time period; the third time period belongs to the second sliding processing time window corresponding to the second time period, and the data scheduling time point within the third time period is earlier than the data rebrushing time point within the third time period; the third time period is the next time period of the second time period;
[0171] Among them, the data scheduling module 11 is further configured to:
[0172] When the data processing time reaches the data scheduling time point within the third time period, based on the data calculation node associated with the payment service, through the data calculation node indicated by the service payment system, perform data scheduling processing on the second service payment data to obtain third service status data, and store the third service status data in the service payment database; the third service status data is used to represent that the service status of the payment service initiated by the second service object is the third service processing status.
[0173] Among them, the third time period includes a first sub-time period and a second sub-time period; the first sub-time period is the next time period of the second time period, and the second sub-time period is the next time period of the first sub-time period; the data rebrushing time point within the third time period includes the first data rebrushing time point within the first sub-time period; the data backbrushing link is determined when the data processing time reaches the first data rebrushing time point;
[0174] Among them, the data backbrushing module 13 further includes: a second rebrushing unit 132;
[0175] The second rebrushing unit 132 is configured to:
[0176] Determine the service payment data associated with the payment service generated within the second time period as the second service payment data;
[0177] Perform data rebrushing processing on the first service payment data and the second service payment data based on the data backbrushing link to obtain first sub-service status data, and determine the second service status data based on the first sub-service status data.
[0178] Among them, the third time period includes a first sub-time period and a second sub-time period; the first sub-time period is the next time period of the second time period, and the second sub-time period is the next time period of the first sub-time period; the data reflashing time points within the third time period include a first data reflashing time point within the first sub-time period and a second data reflashing time point within the second sub-time period; the second service status data includes first sub-service status data; the first sub-service status data is used to characterize that the service status of the payment service initiated by the first service object is the first sub-processing status; the first sub-service status data is determined based on the first service payment data and the service payment data generated within the second time period when the data processing time reaches the first data reflashing time point; the data backflashing link is determined when the data processing time reaches the second data reflashing time point.
[0179] The second reflashing unit 132 is further configured to:
[0180] Determine the service payment data associated with the payment service generated within the second time period and the service payment data associated with the payment service generated within the first sub-time period as the second service payment data;
[0181] Perform data reflashing processing on the first service payment data and the second service payment data based on the data backflashing link to obtain second sub-service status data, and determine the second service status data based on the first sub-service data status data and the second sub-service status data.
[0182] Among them, the second service payment data includes the service payment data generated by the third service object within the first sub-time period;
[0183] Among them, the data scheduling module 11 is further configured to:
[0184] When the data processing time reaches the data scheduling time point within the second sub-time period, perform data scheduling processing on the service payment data generated within the first sub-time period based on the data calculation node associated with the payment service and the data calculation node indicated by the service payment system to obtain fourth service status data, and store the fourth service status data in the service payment database; the fourth service status data is used to characterize that the service status of the payment service initiated by the third service object is the fourth service processing status; the second sub-time period belongs to the third sliding processing time window corresponding to the first sub-time period, and the data scheduling time point within the second sub-time period is earlier than the data reflashing time point within the second sub-time period.
[0185] Among them, the first service status data includes first status data for the target payment service; the first status data includes the service entity information of the target payment service and N service association information corresponding to the service entity information; N is a positive integer; the first status data is determined based on the service payment data for the target payment service in the first service payment data;
[0186] Among them, the data rebrushing module 13 further includes: an associated data determination unit 133;
[0187] Among them, the associated data determination unit 133 is used for:
[0188] Perform data rebrushing processing on the service payment data to be rebrushed to obtain M service association information associated with the service entity information; M is a positive integer greater than or equal to N;
[0189] Determine the second status data associated with the target payment service based on the service entity information and the M service association information, and determine the second service status data based on the second status data.
[0190] Among them, the data format of the first service status data is the first data format; the data calculation node includes a data summary node and a data import node;
[0191] Among them, the data scheduling module 11 includes: a data import unit 111;
[0192] Among them, the data import unit 111 is used for:
[0193] Perform data summary processing on the first service payment data through the data summary node to obtain initial service status data, and store the initial service status data in the data warehouse; the data format of the initial service status data is the second data format; the second data format is different from the first data format;
[0194] Perform data import processing on the initial service status data in the data warehouse through the data import node to obtain the first service status data.
[0195] Among them, the initial service state data includes the initial state data for the target payment service; the data format of the initial state data is the second data format; the initial state data includes the initial service entity information of the target payment service and N initial service association information corresponding to the initial service entity information; N is a positive integer; in the initial state data, each of the N initial service association information is stored in a one-to-one association with the initial service entity information, and the storage times of the initial service entity information is N; each of the initial service association information includes at least one association field information; the initial service entity information includes at least one entity field information;
[0196] Among them, the data import unit 111 is used for:
[0197] Based on each entity field information included in the initial service entity information, respectively determine a entity key value information, obtaining at least one entity key value information; one entity field information corresponds to one entity key value information; the at least one entity key value information is used to indicate the service entity information of the target payment service;
[0198] Determine at least one association field information in each of the initial service association information as an association data value information, and determine the association key value information corresponding to each of the initial service association information based on the association data value information corresponding to each of the initial service association information; the association data value information of one initial service association information corresponds to one association key value information; the association key value information corresponding to each of the initial service association information is used to indicate the service association information of the target payment service;
[0199] Through the data import node, perform data import processing on the at least one entity key value information in the data warehouse and the association key value information corresponding to each of the initial service association information, obtaining the first state data corresponding to the target payment service; the data format of the first state data is the first data format;
[0200] Determine the first service state data based on the first state data.
[0201] Among them, the back brush link determination module 12 includes: a topology graph determination unit 121;
[0202] Among them, the topology graph determination unit 121 is used for:
[0203] Obtain a computing node topology graph associated with the service payment system; the computing node topology graph includes computing nodes and edges; one computing node is used to indicate the computing logic for data processing; one edge is determined based on the computing order between the two connected computing nodes;
[0204] Determine a computing node that matches the data computing node in the computing node topology diagram, and determine a data backflush link for changing the first service processing state based on the computing node that matches the data computing node and the connection edge between the computing nodes that match the data computing node.
[0205] Among them, the data processing device 1 further includes: a jump window module 15;
[0206] Among them, the jump window module 15 is used for:
[0207] When the data processing time reaches the data reflush time point within the fourth time period, determine a data backflush link for changing the second service processing state based on the data computing node associated with the payment service; the fourth time period belongs to the jump time window corresponding to the first time period; the jump time window is later than the last time period within the first sliding processing time window;
[0208] Perform data reflush processing on the first service payment data, the second service payment data, and the third service payment data based on the data backflush link to obtain fifth service status data; the third service payment data includes service payment data associated with the payment service generated by an associated object associated with the first service object in the previous time period of the fourth time period; the fifth service status data is used to characterize that the service status of the payment service is the fifth service processing state;
[0209] When updating the second service status data in the service payment database based on the fifth service status data, update the service status of the payment service from the second service processing state to the fifth service processing state.
[0210] Among them, the backflush link determination module 12 includes: a reflush time point determination unit 122;
[0211] Among them, the reflush time point determination unit 122 is used for:
[0212] Obtain the resource occupancy information of the service processing device associated with the data computing node, and determine the data reflush time point within the third time period based on the resource occupancy information.
[0213] Please refer to Figure 20 , Figure 20 is a schematic structural diagram of a computer device provided by an embodiment of the present application. As Figure 20As shown in the figure, the computer device 1000 may include: a processor 1001, a network interface 1004, and a memory 1005. In addition, the computer device 1000 may further include: a user interface 1003 and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display) and a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. Optionally, the memory 1005 may also be at least one storage device located far from the aforementioned processor 1001. As Figure 20 shown, the memory 1005, as a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.
[0214] In the Figure 20 computer device 1000 as shown in the figure, the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an input interface for users; and the processor 1001 can be used to call the device control application program stored in the memory 1005 to execute the description of the data processing method in any of the foregoing corresponding embodiments, which will not be elaborated here. In addition, the description of the beneficial effects of using the same method will not be elaborated either.
[0215] In addition, it should be noted here that: The embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the aforementioned data processing device 1, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the description of the data processing method in the foregoing embodiments. Therefore, it will not be elaborated here. In addition, the description of the beneficial effects of using the same method will not be elaborated either. For the technical details not disclosed in the embodiment of the computer-readable storage medium involved in the present application, please refer to the description of the method embodiment of the present application.
[0216] The above computer-readable storage medium may be the data processing device provided in any of the foregoing embodiments or the internal storage unit of the above computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit and the external storage device of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store the data that has been output or is to be output.
[0217] In addition, it should be noted here that: The embodiments of the present application also provide a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in any of the foregoing corresponding embodiments. In addition, the description of the beneficial effects of the same method will not be repeated. For the technical details not disclosed in the computer program product or the computer program embodiments of the present application, please refer to the description of the method embodiments of the present application.
[0218] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit that includes the function of the module or unit.
[0219] In the description of the embodiments of the present application, the terms "first", "second", etc. in the specification, claims and drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally further includes steps or modules not listed, or optionally further includes other step units inherent to these processes, methods, devices, products or equipment.
[0220] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.
[0221] The above-disclosed are only the preferred embodiments of this application, and of course, they cannot be used to limit the scope of rights of this application. Therefore, equivalent changes made according to the claims of this application still fall within the scope covered by this application.
Claims
1. A data processing method, characterized in that, The method includes: When the data processing time reaches the data scheduling time point within the second time period, the data calculation node indicated by the business payment system is used to perform data scheduling processing on the first business payment data generated by the first business object within the first time period, so as to obtain first business status data, and store the first business status data in the business payment database; the first time period is the previous time period of the second time period; the second time period belongs to the first sliding processing time window corresponding to the first time period; the first business status data is used to characterize that the business status of the payment business initiated by the first business object is the first business processing status; When the data processing time reaches the data rebrushing time point within the third time period in the first sliding processing time window, a data rebrushing link for changing the first business processing status is determined based on the data calculation node associated with the payment business; the third time period is any time period in the first sliding processing time window that is later than the second time period; Based on the data rebrushing link, data rebrushing processing is performed on the first business payment data and the second business payment data to obtain second business status data; the second business payment data includes business payment data associated with the payment business generated by an associated object associated with the first business object within the second time period to the third time period; the second business status data is used to characterize that the business status of the payment business is the second business processing status; When updating the first business status data in the business payment database based on the second business status data, the business status of the payment business is updated from the first business processing status to the second business processing status.
2. The method according to claim 1, wherein The third time period is the next time period of the second time period; The performing data rebrushing processing on the first business payment data and the second business payment data based on the data rebrushing link to obtain second business status data includes: Determining the business payment data associated with the payment business generated within the second time period as the second business payment data; Performing data rebrushing processing on the first business payment data and the second business payment data based on the data rebrushing link to obtain the second business status data.
3. The method according to claim 1, characterized in that The second business payment data includes business payment data generated by a second business object within the second time period; the third time period belongs to the second sliding processing time window corresponding to the second time period, and the data scheduling time point within the third time period is earlier than the data rebrushing time point within the third time period; The third time period is the next time period of the second time period; The method further includes: When the data processing time reaches the data scheduling time point within the third time period, based on the data calculation node associated with the payment service, through the data calculation node indicated by the business payment system, perform data scheduling processing on the second business payment data to obtain third business status data, and store the third business status data in the business payment database; the third business status data is used to represent that the business status of the payment service initiated by the second business object is the third business processing status.
4. The method according to claim 1, characterized in that, The third time period includes a first sub-time period and a second sub-time period; the first sub-time period is the next time period of the second time period, and the second sub-time period is the next time period of the first sub-time period; the data rebrushing time point within the third time period includes the first data rebrushing time point within the first sub-time period; the data backbrushing link is determined when the data processing time reaches the first data rebrushing time point; The data rebrushing process for the first business payment data and the second business payment data based on the data backbrushing link to obtain the second business status data includes: Determine the business payment data associated with the payment service generated within the second time period as the second business payment data; Perform data rebrushing processing on the first business payment data and the second business payment data based on the data backbrushing link to obtain first sub-business status data, and determine the second business status data based on the first sub-business status data.
5. The method according to claim 1, wherein The third time period includes a first sub-time period and a second sub-time period; the first sub-time period is the next time period of the second time period, and the second sub-time period is the next time period of the first sub-time period; the data rebrushing time points within the third time period include the first data rebrushing time point within the first sub-time period and the second data rebrushing time point within the second sub-time period; the second business status data includes first sub-business status data; the first sub-business status data is used to represent that the business status of the payment service initiated by the first business object is the first sub-business processing status; the first sub-business status data is determined based on the first business payment data and the business payment data generated within the second time period when the data processing time reaches the first data rebrushing time point; the data backbrushing link is determined when the data processing time reaches the second data rebrushing time point; The data rebrushing process for the first business payment data and the second business payment data based on the data backbrushing link to obtain the second business status data includes: Determine the business payment data associated with the payment service generated within the second time period and the business payment data associated with the payment service generated within the first sub-time period as the second business payment data; Based on the data rebrushing link, perform data rebrushing processing on the first service payment data and the second service payment data to obtain second sub-service status data, and determine the second service status data based on the first sub-service data status data and the second sub-service status data.
6. The method according to claim 5, wherein The second service payment data includes service payment data generated by a third service object within the first sub-time period. The method further includes: When the data processing time reaches the data scheduling time point within the second sub-time period, based on the data calculation nodes associated with the payment service and through the data calculation nodes indicated by the service payment system, perform data scheduling processing on the service payment data generated within the first sub-time period to obtain fourth service status data, and store the fourth service status data in the service payment database; the fourth service status data is used to represent that the service status of the payment service initiated by the third service object is the fourth service processing status; the second sub-time period belongs to the third sliding processing time window corresponding to the first sub-time period, and the data scheduling time point within the second sub-time period is earlier than the data rebrushing time point within the second sub-time period.
7. The method according to claim 1, wherein The first service status data includes first status data for a target payment service; the first status data includes service subject information of the target payment service and N service association information corresponding to the service subject information; N is a positive integer; the first status data is determined based on the service payment data for the target payment service in the first service payment data. The performing data rebrushing processing on the first service payment data and the second service payment data based on the data rebrushing link to obtain second service status data includes: Based on the first service payment data and the second service payment data, determine the service payment data to be rebrushed. Perform data rebrushing processing based on the service payment data to be rebrushed to obtain M service association information associated with the service subject information; M is a positive integer greater than or equal to N. Based on the service subject information and the M service association information, determine the second status data associated with the target payment service, and determine the second service status data based on the second status data.
8. The method according to claim 1, characterized in that, The data format of the first service status data is the first data format; the data calculation nodes include a data summary node and a data import node. The performing data scheduling processing on the first service payment data generated by a first service object within a first time period through the data calculation nodes indicated by the service payment system to obtain first service status data includes: Perform data summary processing on the first service payment data through the data summary node to obtain initial service status data, and store the initial service status data in the data warehouse; the data format of the initial service status data is the second data format; the second data format is different from the first data format. The initial business status data in the data warehouse is imported through the data import node to obtain the first business status data.
9. The method according to claim 8, characterized in that, The initial business status data includes the initial status data for the target payment service; the data format of the initial status data is the second data format; the initial status data includes the initial business entity information of the target payment service and N initial business association information corresponding to the initial business entity information; N is a positive integer; In the initial status data, each of the N initial business association information is stored in a one-to-one association with the initial business entity information, and the storage times of the initial business entity information is N; each of the initial business association information includes at least one association field information; the initial business entity information includes at least one entity field information; The step of importing the initial business status data in the data warehouse through the data import node to obtain the first business status data includes: Based on each entity field information included in the initial business entity information, a corresponding entity key value information is determined respectively, to obtain at least one entity key value information; one entity field information corresponds to one entity key value information; the at least one entity key value information is used to indicate the business entity information of the target payment service; At least one association field information in each initial business association information is determined as an association data value information, and based on the association data value information corresponding to each initial business association information, the association key value information corresponding to each initial business association information is determined; one association data value information of an initial business association information corresponds to one association key value information; the association key value information corresponding to each initial business association information is used to indicate the business association information of the target payment service; The data import node performs data import processing on the at least one entity key value information and the association key value information corresponding to each initial business association information in the data warehouse to obtain the first status data corresponding to the target payment service; the data format of the first status data is the first data format; The first business status data is determined based on the first status data.
10. The method according to claim 1, characterized in that, Determining a data rollback link for changing the first business processing status based on a data calculation node associated with the payment service includes: Obtaining a calculation node topology diagram associated with the business payment system; the calculation node topology diagram includes calculation nodes and edges; one calculation node is used to indicate the calculation logic for data processing; one edge is determined based on the calculation order between the two connected calculation nodes; In the calculation node topology diagram, a calculation node matching the data calculation node is determined, and based on the calculation node matching the data calculation node and the edge between the calculation nodes matching the data calculation node, a data rollback link for changing the first business processing status is determined.
11. The method according to claim 1, wherein The method further includes: When the data processing time reaches the data rebrushing time point within the fourth time period, determine a data rebrushing link for changing the second service processing state based on a data calculation node associated with the payment service; the fourth time period belongs to a jump time window corresponding to the first time period; the jump time window is later than the last time period within the first sliding processing time window; Perform data rebrushing processing on the first service payment data, the second service payment data, and the third service payment data based on the data rebrushing link to obtain fifth service status data; the third service payment data includes service payment data associated with the payment service generated by an associated object associated with the first service object within the previous time period of the fourth time period; the fifth service status data is used to represent that the service status of the payment service is the fifth service processing state; When updating the second service status data in the service payment database based on the fifth service status data, update the service status of the payment service from the second service processing state to the fifth service processing state.
12. The method according to claim 1, wherein The method further includes: Obtain resource occupancy information of a service processing device associated with the data calculation node, and determine the data rebrushing time point within the third time period based on the resource occupancy information.
13. A data processing device, characterized in that, The device includes: A data scheduling module, configured to, when the data processing time reaches the data scheduling time point within the second time period, perform data scheduling processing on the first service payment data generated by a first service object within the first time period through a data calculation node indicated by the service payment system to obtain first service status data, and store the first service status data in the service payment database; the first time period is the previous time period of the second time period; the second time period belongs to a first sliding processing time window corresponding to the first time period; the first service status data is used to represent that the service status of the payment service initiated by the first service object is the first service processing state; A rebrushing link determination module, configured to, when the data processing time reaches the data rebrushing time point within the third time period in the first sliding processing time window, determine a data rebrushing link for changing the first service processing state based on a data calculation node associated with the payment service; the third time period is any time period in the first sliding processing time window that is later than the second time period; A data rebrushing module, configured to perform data rebrushing processing on the first service payment data and the second service payment data based on the data rebrushing link to obtain second service status data; the second service payment data includes service payment data associated with the payment service generated by an associated object associated with the first service object within the second time period to the third time period; the second service status data is used to represent that the service status of the payment service is the second service processing state; A data update module, configured to update the first service status data in the service payment database based on the second service status data, and update the service status of the payment service from the first service processing status to the second service processing status.
14. A computer device, characterized in that, It includes a memory and a processor; The memory is connected to the processor. The memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1-12.
15. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium. The computer program is adapted to be loaded and executed by a processor so that a computer device having the processor executes the method according to any one of claims 1-12.
16. A computer program product, characterized in that, It includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the method according to any one of claims 1-12 is implemented.