Data reconciliation processing method, device, computer equipment and storage medium
By performing data reconciliation when business processing fails and using nodes with the lowest load rate for data reconciliation, the problem of high load rate in data synchronization is solved, and the reliability and stability of the system are improved.
Patent Information
- Application Number
- CN202210678419.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-16
- Publication Date
- 2025-08-22
- Estimated Expiration
- 2042-06-16
AI Technical Summary
When synchronizing data between different systems, data inconsistency frequently occurs, resulting in high load rate for data reconciliation processing. How to reduce the load rate has become an urgent problem.
By receiving information synchronized by the target node, the service processing results are judged, and data reconciliation is performed only when the service processing fails. Data reconciliation is performed using the service node with the lowest load rate to reduce the number of data reconciliation processing times and reduce the system load.
It effectively reduces the load rate of data reconciliation processing, improves the reliability and stability of the system, and ensures data consistency.
Smart Images

Figure CN117290435B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of Internet technology, and in particular to a data reconciliation processing method, apparatus, computer equipment, and storage medium. Background Art
[0002] As business scenarios become more diverse and personalized, the need to reduce the impact and coupling between different systems is growing. Systems with long links often have multiple copies of data, meaning that a single piece of data is stored in multiple systems. This requires data synchronization logic between these systems. However, data inconsistencies often occur when data is synchronized between different systems, necessitating reconciliation of data across these systems.
[0003] Currently, data reconciliation primarily involves verifying and checking the sent and received messages within data systems (such as big data access systems) to verify the accuracy and reliability of the data flow. However, due to the large amount of data required for comparison in data systems, the data reconciliation process is subject to high processing load. Therefore, reducing this load is an urgent issue. Summary of the Invention
[0004] Based on this, it is necessary to provide a data reconciliation processing method, device, computer equipment, computer-readable storage medium and computer program product that can reduce the load rate to address the above technical problems.
[0005] In a first aspect, the present application provides a data reconciliation processing method. The method comprises:
[0006] Receiving first synchronization information synchronized by a target node, where the first synchronization information includes a target service request, where the target service request is used to instruct calling target data to process a target service;
[0007] If the second synchronization information synchronized by the target node is received within the preset time interval, the service processing result corresponding to the target service request is determined according to the second synchronization information;
[0008] If the business processing result is a business processing failure, data reconciliation processing is performed on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node.
[0009] In one embodiment, if the business processing result is a business processing failure, data reconciliation processing is performed on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node, including:
[0010] If the business processing result is business processing failure, obtain the number of businesses running on each business service node;
[0011] Based on the number of businesses running on each business service node, calculate the load rate of each business service node in performing data reconciliation processing on the target data of the target node and associated nodes;
[0012] The business service node corresponding to the smallest load rate is determined as the target business service node;
[0013] The target business service node performs data reconciliation on the target data of the target node and the associated nodes.
[0014] In one embodiment, determining a service processing result corresponding to the target service request according to the second synchronization information includes:
[0015] Obtaining a data call result corresponding to the target service request included in the second synchronization information;
[0016] If the data call result includes: data call failure for target data at the target node, then the business processing result is determined to be business processing failure.
[0017] In one embodiment, the failure of the data call for the target data at the target node includes: failing to read the target data from the target node.
[0018] In one embodiment, the failure of a data call to target data at a target node includes: failing to write the target data to the target node.
[0019] In one of the embodiments, the failure of the data call for the target data at the target node includes: failing to delete the target data from the target node.
[0020] In one embodiment, determining a service processing result corresponding to the target service request according to the second synchronization information includes:
[0021] If the data call result corresponding to the target service request does not exist in the second synchronization information, it is determined that the service processing result corresponding to the target service request is a service processing failure.
[0022] In one embodiment, the method further comprises:
[0023] If the second synchronization information synchronized by the target node is not received within the preset time interval, it is determined that the service processing result is a service processing failure.
[0024] In one embodiment, the method further comprises:
[0025] Count the business processing results corresponding to multiple target business requests within a preset time period;
[0026] Based on the business processing results corresponding to each target business request, the business processing results are calculated as the data distribution of business processing failures and business processing successes;
[0027] The evaluation score is calculated based on the data distribution of business processing failures and business processing successes. The evaluation score is used to describe the proportion of data inconsistencies in business processing within a preset time period.
[0028] In a second aspect, the present application further provides a data reconciliation processing device. The device comprises:
[0029] A communication module, configured to receive first synchronization information synchronized by a target node, the first synchronization information including a target service request, the target service request being used to instruct calling target data to process a target service;
[0030] The processing module is used to determine the business processing result corresponding to the target business request based on the second synchronization information received by the communication module from the target node within a preset time interval; if the business processing result is a business processing failure, perform data reconciliation processing on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node.
[0031] In a third aspect, the present application further provides a data reconciliation processing device. The device comprises:
[0032] The configuration management module is used to provide add, delete, modify, and query logic for consistency scenarios and event pool configurations. The event pool includes a consistency event pool and an error event pool. The consistency event pool is used to trigger the reception of the first synchronization information of the target node synchronization. The first synchronization information includes a target service request, which is used to instruct the call of target data to process the target service.
[0033] An event management module is used to manage the consistency event pool; if the second synchronization information of the target node synchronization is received within a preset time interval, the service processing result corresponding to the target service request is determined according to the second synchronization information;
[0034] The engine module is used to perform data reconciliation processing on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node if the business processing result is a business processing failure;
[0035] The error event management module is used to manage the error event pool.
[0036] In a fourth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0037] Receiving first synchronization information synchronized by a target node, where the first synchronization information includes a target service request, where the target service request is used to instruct calling target data to process a target service;
[0038] If the second synchronization information synchronized by the target node is received within the preset time interval, the service processing result corresponding to the target service request is determined according to the second synchronization information;
[0039] If the business processing result is a business processing failure, data reconciliation processing is performed on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node.
[0040] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:
[0041] Receiving first synchronization information synchronized by a target node, where the first synchronization information includes a target service request, where the target service request is used to instruct calling target data to process a target service;
[0042] If the second synchronization information synchronized by the target node is received within the preset time interval, the service processing result corresponding to the target service request is determined according to the second synchronization information;
[0043] If the business processing result is a business processing failure, data reconciliation processing is performed on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node.
[0044] In a sixth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:
[0045] Receiving first synchronization information synchronized by a target node, where the first synchronization information includes a target service request, where the target service request is used to instruct calling target data to process a target service;
[0046] If the second synchronization information synchronized by the target node is received within the preset time interval, the service processing result corresponding to the target service request is determined according to the second synchronization information;
[0047] If the business processing result is a business processing failure, data reconciliation processing is performed on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node.
[0048] The above-mentioned data reconciliation processing method, apparatus, computer equipment, storage medium and computer program product receive first synchronization information synchronized by the target node, the first synchronization information including a target business request, the target business request being used to instruct the call of target data to process the target business. If second synchronization information synchronized by the target node is received within a preset time interval, the business processing result corresponding to the target business request is determined according to the second synchronization information. If the business processing result is a business processing failure, data reconciliation processing is performed on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node. Based on the business processing result corresponding to the target business request, it is determined whether to perform data reconciliation processing. Since data reconciliation processing only needs to be performed on the target data indicated by the target business request when the business processing result is a business processing failure, the number of data reconciliation processing times can be reduced to reduce the load rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 A schematic diagram of the system architecture of a data sharing system in one embodiment;
[0050] Figure 2 Schematic diagram of the architecture of a blockchain composed of blocks in one embodiment;
[0051] Figure 3 1. A schematic diagram of a new block generation process in a blockchain according to one embodiment;
[0052] Figure 4 A schematic diagram of an application scenario of a data reconciliation processing method in one embodiment;
[0053] Figure 5 A schematic diagram of an application system of a data reconciliation processing method in one embodiment;
[0054] Figure 6 A schematic diagram of the architecture of functional modules in an application system of a data reconciliation processing method in one embodiment;
[0055] Figure 7 1 is a flow chart of a data reconciliation processing method according to an embodiment;
[0056] Figure 8 A flowchart of a data reconciliation processing method according to another embodiment;
[0057] Figure 9 A schematic diagram illustrating determining a load rate and performing data reconciliation processing through a target business service node in one embodiment;
[0058] Figure 10 A schematic diagram of a portion of a process for determining a business processing result in one embodiment;
[0059] Figure 11A schematic diagram of a portion of a process for determining a business processing result in another embodiment;
[0060] Figure 12 A schematic diagram of a scenario recommended by the article in one embodiment;
[0061] Figure 13 A schematic diagram of a portion of the process for determining a business processing result in yet another embodiment;
[0062] Figure 14 A partial flow chart of determining a business processing result in yet another embodiment;
[0063] Figure 15 Schematic diagram of a data reconciliation processing method in another embodiment;
[0064] Figure 16 A schematic diagram of an interface displaying evaluation scores in one embodiment;
[0065] Figure 17 A schematic diagram of the overall process of a data reconciliation processing method in one embodiment;
[0066] Figure 18 This is a structural block diagram of a data reconciliation processing device in one embodiment;
[0067] Figure 19 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0068] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.
[0069] Before explaining the data reconciliation processing method provided in the embodiment of the present application, the relevant contents of the data sharing system to which the embodiment of the present application is applied are first explained.
[0070] The data reconciliation processing method provided in the embodiment of the present application can be applied to Figure 1The data sharing system shown, the data sharing system 100 refers to a system for sharing data between nodes, and the data sharing system may include multiple nodes 101, and the multiple nodes 101 may refer to each client in the data sharing system. Each node 101 can receive input information when performing normal work, and maintain the shared data in the data sharing system based on the received input information. In order to ensure the intercommunication of information in the data sharing system, there can be an information connection between each node in the data sharing system, and information can be transmitted between nodes through the above-mentioned information connection. For example, when any node in the data sharing system receives input information, the other nodes in the data sharing system obtain the input information according to the consensus algorithm, and store the input information as data in the shared data, so that the data stored on all nodes in the data sharing system are consistent.
[0071] Each node in the data sharing system has a corresponding node identifier, and each node in the data sharing system can store the node identifiers of other nodes in the data sharing system so that the generated blocks can be broadcast to other nodes in the data sharing system based on the node identifiers of other nodes. Each node can maintain a node identifier list as shown in the following table, and store the node name and node identifier in the node identifier list accordingly. Among them, the node identifier can be the Internet Protocol (IP) address of the interconnection protocol between networks or any other information that can be used to identify the node. Table 1 only uses the IP address as an example for explanation.
[0072] Table 1
[0073] Node Name Node ID Node 1 117.114.151.174 Node 2 117.116.189.145 … … Node N 119.123.789.258
[0074] Each node in the data sharing system stores the same blockchain. The blockchain consists of multiple blocks, see Figure 2 The blockchain consists of multiple blocks. The genesis block includes a block header and a block body. The block header stores the input information feature value, version number, timestamp and difficulty value, and the block body stores the input information; the next block of the genesis block uses the genesis block as the parent block, and the next block also includes a block header and a block body. The block header stores the input information feature value of the current block, the block header feature value, version number, timestamp and difficulty value of the parent block, and so on, so that the block data stored in each block in the blockchain is associated with the block data stored in the parent block, ensuring the security of the input information in the block.
[0075] When generating each block in the blockchain, see Figure 3When the node where the blockchain is located receives the input information, it verifies the input information. After the verification is completed, the input information is stored in the memory pool and the hash tree used to record the input information is updated. After that, the update timestamp is updated to the time when the input information is received, and different random numbers are tried to calculate the eigenvalue multiple times so that the calculated eigenvalue can satisfy formula (1):
[0076] SHA256(SHA256(version+prev_hash+merkle_root+ntime+nbits+x)) <TARGET;(1)
[0077] Among them, SHA256 is the eigenvalue algorithm used to calculate the eigenvalue, version number (version) is the version information of the relevant block protocol in the blockchain, prev_hash is the block header eigenvalue of the parent block of the current block, merkle_root is the eigenvalue of the input information, ntime is the update time of the update timestamp, nbits is the current difficulty, which is a fixed value within a period of time and is determined again after exceeding a fixed time interval, x is a random number, and TARGET is the eigenvalue threshold, which can be determined based on nbits.
[0078] In this way, when a random number that satisfies the above formula is calculated, the information can be stored accordingly, and the block header and block body can be generated to obtain the current block. Subsequently, the blockchain node sends the newly generated block to other nodes in the data sharing system based on the node identifiers of other nodes in the data sharing system. The other nodes verify the newly generated block and, after verification, add the newly generated block to their stored blockchain.
[0079] It should be understood that in the scenario applied to the blockchain, the data stored in each node remains consistent. For example, in the scenario specifically applied to article recommendation, the article data published by the publisher will first be stored in the associated node, and then pushed to the target node for storage through the associated node, so that when the user uses the terminal to perform operations such as querying or browsing, the article data will be obtained from the storage of the target node and displayed on the display interface of the terminal. Based on this, if the publisher of the article data has deleted the article data, the associated node and the target node both need to delete the article data to ensure that the data stored in each node is consistent. In actual applications, there are also scenarios where the target node reads the target data and writes the target data in the associated node. For example, in the scenario specifically applied to remittances, the remitter's account corresponds to the target node, and the remittance recipient's account corresponds to the associated node. Based on this, the remitter reads the target value data from the target node and writes the target value data in the associated node during the process of transferring the target value data to the remittance recipient. Therefore, the above examples should not be understood as limitations of this application.
[0080] The data reconciliation processing method provided in this application can be applied to Figure 4 In the application environment shown. Figure 4 It can be seen that the data reconciliation processing method, when applied to the data reconciliation processing node 402 included in the blockchain network, can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Secondly, the blockchain network also includes various blockchain nodes 404, which can specifically be smartphones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, in-vehicle terminals, smart TVs, etc., but are not limited to these. Blockchain node 404 can also be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. In particular, when the data reconciliation processing node 402 receives a first synchronization message synchronized by the blockchain node 404, if a second synchronization message synchronized by the blockchain node 404 is received within a preset time interval, the business processing result corresponding to the target business request can be determined based on the second synchronization message. If the business processing result is a business processing failure, data reconciliation processing can be performed on the target data of the blockchain node 404 and the associated nodes (also blockchain nodes) that have a reconciliation relationship with the blockchain node 404.
[0081] Specifically, the data reconciliation processing method in this embodiment includes three process stages: data preparation, reconciliation logic, and reconciliation processing. The following first introduces the above three process stages respectively:
[0082] 1. Data Preparation
[0083] Data preparation must support simple and rapid access to data from different business services to trigger and drive the subsequent operation of the reconciliation system. Because different business services have different data characteristics, the reconciliation system should support multiple forms of data access.
[0084] Specifically, the forms of data access may include but are not limited to data push and data pull. Data push includes online data push and offline data push. Online data push is specifically based on the online business service call interface through the Remote Procedure Call Protocol (RPC), HyperText Transfer Protocol (HTTP), etc., to push the target data that needs to be reconciled, while offline push is specifically based on the offline business service through file upload to push the target data that needs to be reconciled. Secondly, data pull is specifically provided by the business service with files that meet the standards of the reconciliation system, so that the reconciliation system actively pulls the target data that needs to be reconciled.
[0085] 2. Reconciliation Logic
[0086] Reconciliation logic is a set of customized reconciliation rules for business services. These rules are determined based on the specific business service. The results of these rules can determine subsequent reconciliation processing, callback notifications, and business alerts. Furthermore, reconciliation logic must not only deliver accurate results but also support flexible configuration and dynamic adjustment based on specific business reconciliation requirements.
[0087] 3. Error Handling
[0088] Error handling specifically processes target data that has failed reconciliation processing, and generally includes functions such as retry, notification, alarm, display, and even direct repair for inconsistent results.
[0089] It should be understood that Figure 5 As shown, in actual applications, the business system first determines that the target data needs to be called through the target business request. Before calling the target data to execute the business logic, the business system needs to report the business event to the consistency system first, and then call the target data. This process involves multiple systems and multiple storages, so it may fail at any step in the figure. Therefore, the consistency system processes the updated business event status reported by the business system to compare the consistency of the target data (that is, the same data) called in each storage (that is, data reconciliation processing), so as to discover inconsistent data and repair it.
[0090] In this embodiment, the general reconciliation system is abstracted into a configuration management module, an event management module, an engine module and an error event management module, thereby obtaining a data reconciliation processing device of this embodiment. Figure 6 As shown, the specific functions of each module in the data reconciliation processing device are introduced below.
[0091] The configuration management module 602 is used to provide the add, delete, modify and query logic for the consistency scenario configuration and the event pool configuration. The event pool includes a consistency event pool and an error event pool. Among them, the consistency event pool is used to trigger the first synchronization information of the target node synchronization. The first synchronization information includes a target business request. The target business request is used to indicate the call of the target data to process the target business. Secondly, the configuration management module 602 is also used to provide a user management interaction interface. Among them, the consistency scenario is specifically an entity that performs data reconciliation processing. The consistency scenario includes but is not limited to a variety of configuration information for data reconciliation processing of the target data: business scenario name, subscribed reconciliation event pool, reconciliation event delayed consumption time, target data acquisition interface information, reconciliation judgment logic (rules or scripts) and reconciliation processing information, etc., wherein the interface information specifically includes the interface information of the benchmark party and the detected party, and the reconciliation processing information specifically includes but is not limited to retry strategy, callback notification information, etc.
[0092] The event management module 604 is used to manage the consistency event pool. In addition, if the second synchronization information of the target node synchronization is received within a preset time interval, the event management module 604 determines the business processing result corresponding to the target business request based on the second synchronization information. Secondly, the event management module 604 is also used to provide a variety of data access methods, and after the data format of the target data is adapted, the speed limit of the target data with the adapted data format is written into different reconciliation event pools. Secondly, the management of the consistency event pool is specifically the management functions of the consistency event pool, such as sub-table management (sub-table according to different event sources), partition management, and life cycle management.
[0093] The engine module 606 is used to perform data reconciliation processing on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node if the business processing result is a business processing failure. Specifically, it includes but is not limited to reconciliation event acquisition (i.e., performing data reconciliation processing on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node), data pulling (i.e., obtaining target data), reconciliation logic execution, and reconciliation result processing.
[0094] The error event management module 608 is used to manage the error event pool. Secondly, it is also used to provide basic read and write functions such as writing and storing error events (events where reconciliation fails), pulling error tasks that have met the retry conditions, etc. Secondly, the management of the error event pool specifically includes management functions such as table management (based on different reconciliation scenarios, etc.), partition management (based on daily partitions, etc.), and life cycle management of the error event pool. In addition, the error event management module 608 also provides various types of error event callback notification functions, and the error event management module 608 can also provide user message push and alarm functions.
[0095] Next, the data reconciliation processing method provided in the embodiment of the present application is explained in detail. In one embodiment, Figure 7 As shown, a data reconciliation processing method is provided, which is applied to Figure 4 Taking the data reconciliation processing node 402 in
[15] as an example, the following steps are included:
[0096] Step 702: Receive first synchronization information synchronized by a target node. The first synchronization information includes a target service request. The target service request is used to instruct to call target data to process a target service.
[0097] The target service request is used to instruct the invocation of target data to process a target service. Specifically, the target service requires the target node and associated nodes to jointly invoke the target data. For example, the target service may be a service that reads target data from the target node and writes it to associated nodes. Alternatively, the target service may be a service that writes target data to the target node and associated nodes. Alternatively, the target service may be a service that deletes target data from the target node and associated nodes, and so on. This is not limited here.
[0098] Based on this, when a user initiates a target business that requires the target node and the associated nodes to jointly call the target data, the target node and the associated nodes will receive the aforementioned target business request, and the target node and the associated nodes will synchronize the synchronization information including the target business request, so that the data reconciliation processing node receives the first synchronization information synchronized by the target node, and also receives the synchronization information sent by the associated node, so that the data reconciliation processing node can determine that the target node and the associated nodes need to jointly call the target data to perform the target business. Specifically, the aforementioned target node and the associated nodes are both Figure 4 The blockchain nodes introduced and the associated nodes can be one or more blockchain nodes.
[0099] Step 704: If the second synchronization information synchronized by the target node is received within the preset time interval, the service processing result corresponding to the target service request is determined according to the second synchronization information.
[0100] Among them, the business processing result is specifically used to describe the result of each node processing the target business based on the target business request. Therefore, if the data reconciliation processing node receives the second synchronization information synchronized by the target node within the preset time interval, the business processing result corresponding to the target business request is determined based on the second synchronization information. The starting time of the aforementioned preset time interval is specifically the time when the first synchronization information is received, and the preset time interval can be 5 seconds (s), 8s or 12s, etc. The specific time length of the preset time interval needs to be determined based on actual conditions and multiple test results, and is not limited here.
[0101] Step 706 : If the business processing result is business processing failure, data reconciliation processing is performed on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node.
[0102] The business processing result can be either a business processing failure or a business processing success. Based on this, if the business processing result determined in step 704 is a business processing failure, the business processing failure may include the target node failing to call the target data based on the target business, i.e., indicating that the target node has not completed the target business. In this case, a reconciliation relationship exists between the target node and the associated nodes, i.e., the data reconciliation processing node needs to perform data reconciliation processing on the target data of the target node and the associated nodes. It should be understood that the business processing result of business processing failure can also include other possibilities, which are not limited here.
[0103] Furthermore, after reconciling the target data of the target node and the associated nodes, if inconsistencies are found in the target data, the inconsistent target data needs to be repaired. Specifically, the error event management module 608 described in the above embodiment is used to repair the inconsistent target data, and the specific details are not limited here.
[0104] In the above-mentioned data reconciliation processing method, whether to perform data reconciliation processing is determined based on the business processing result corresponding to the target business request. Since data reconciliation processing is only required for the target data called by the target business request when the business processing result is a business processing failure, the number of data reconciliation processing times can be reduced to reduce the load rate.
[0105] In the scenario where a large amount of data is being processed for data reconciliation, further consideration needs to be given to how to balance the load of data reconciliation processing on each business service node. Figure 8 As shown, in step 706, if the business processing result is a business processing failure, data reconciliation processing is performed on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node, including:
[0106] Step 802: If the service processing result is service processing failure, the number of services running on each service node is obtained.
[0107] The number of transactions refers to the number of transactions performed when the business service node performs data reconciliation processing on the target data of the target node and associated nodes. Based on this, if the business processing result is determined to be a business processing failure, it means that the target node failed to call the target data based on the target transaction, that is, the target transaction was not completed. At this time, the data reconciliation processing node can determine that data reconciliation processing is required for the target data. Based on this, considering how to balance the load of each business service node, the data reconciliation processing node needs to obtain the number of transactions performed when each business service node performs data reconciliation processing on the target data of the target node and associated nodes before performing data reconciliation processing.
[0108] Step 804 : Based on the number of services running on each business service node, the load rate of each business service node in performing data reconciliation processing on the target data of the target node and the associated nodes is calculated.
[0109] The load rate is specifically used to describe the load rate of the corresponding business service node when performing data reconciliation processing, that is, the ratio of the actual number of businesses when the business service node performs data reconciliation processing to the maximum number of businesses that the business service node can carry.
[0110] Based on this, the data reconciliation processing node calculates the load rate of the business service node when performing data reconciliation processing on the target node and the associated node based on the number of businesses running on each business service node and the reconciliation business determined to perform data reconciliation processing on the target data of the target node and the associated node.
[0111] Specifically, the data reconciliation processing node calculates the aforementioned load rate using formula (2):
[0112]
[0113] in, is the number of services running on the service node at time t, The load rate of the business service node when performing data reconciliation processing on data j.
[0114] Step 806: Determine the business service node corresponding to the load rate with the smallest value as the target business service node.
[0115] The target business service node is the service node that performs data reconciliation processing on the target data of the target node and associated nodes. Specifically, the data reconciliation processing node sorts the load rates of each business service node performing data reconciliation processing on the target data of the target node and associated nodes from high to low based on the load rates, and determines the business service node with the lowest load rate as the target business service node.
[0116] For example, the load rate of business service node A1 for data reconciliation processing of target data is 60%, the load rate of business service node A2 for data reconciliation processing of target data is 75%, and the load rate of business service node A3 for data reconciliation processing of target data is 80%. It can be determined that the load rate of 60% is the load rate with the minimum value, so the data reconciliation processing node can determine that the business service node A1 corresponding to this load rate is the target business service node.
[0117] If there are multiple data that need to be reconciled at the same time, the data reconciliation processing node needs to judge the load rate of each business service node in real time and adjust the business service node for data reconciliation processing. Figure 9 As shown, when reconciliation task 1, reconciliation task 2 and reconciliation task 3 are required, and reconciliation task 1 is specifically to perform data reconciliation processing on data 1, reconciliation task 2 is specifically to perform data reconciliation processing on data 2, and reconciliation task 3 is specifically to perform data reconciliation processing on data 3.
[0118] The data reconciliation processing node is based on distributed lock 1, and can determine that business service node 1 and business service node 2 can execute reconciliation task 1 at this time. Then, based on the aforementioned formula (2), the load rate of business service node 1 and the load rate of business service node 2 are calculated respectively, and the business service node corresponding to the smallest load rate is determined as the target business service node for executing reconciliation task 1.
[0119] Similarly, the data reconciliation processing node determines, based on distributed lock 2, that business service node 1 and business service node 2 can now execute reconciliation task 2, and then, in a similar manner, determines the target business service node for executing reconciliation task 2. Furthermore, the data reconciliation processing node determines, based on distributed lock 3, that business service node 1 and business service node 2 can now execute reconciliation task 3, and then, in a similar manner, determines the target business service node for executing reconciliation task 3.
[0120] Step 808: Perform data reconciliation processing on the target data of the target node and the associated nodes through the target business service node.
[0121] Specifically, after the data reconciliation processing node determines the target business service node in step 806, it will perform data reconciliation processing on the target data of the target node and the associated nodes through the target business service node. The method for performing data reconciliation processing is similar to step 706 and will not be repeated here.
[0122] Further, please refer again to Figure 9 ,like Figure 9As shown, business service node 1 executes reconciliation task 1 and reconciliation task 3 at time 1, while business service node 2 executes reconciliation task 2 at time 1. Secondly, business service node 1 executes reconciliation task 2 at time 2, while business service node 2 executes reconciliation task 1 and reconciliation task 3 at time 2. Business service node 1 executes reconciliation task 1 and reconciliation task 3 at time 3, while business service node 2 executes reconciliation task 2 at time 3. This shows that in actual applications, data reconciliation processing can be performed on the target data of the target node and associated nodes through different business service nodes at different times.
[0123] In this embodiment, taking into account the load balancing of data reconciliation processing on each business service node, the service node corresponding to the minimum load rate during data reconciliation processing is determined as the target service node, so that data reconciliation processing is performed through the target business service node to avoid the problem of load imbalance caused by the heavy load of a single business service node in the scenario where there is a large amount of data to be processed, so as to improve the reliability and stability of the entire data reconciliation processing system, thereby further improving the reliability of data reconciliation processing.
[0124] This embodiment provides multiple methods for determining business processing results, which are described below:
[0125] In one embodiment, Figure 10 As shown, step 704, determining the service processing result corresponding to the target service request according to the second synchronization information, includes:
[0126] Step 902: Obtain the data call result corresponding to the target service request included in the second synchronization information.
[0127] The "data call result" specifically describes the results of each node's call to the target data based on the target service. Upon receiving the target service request, the target node will call the target data based on the target service request to process the target service, obtain the data call result corresponding to the target service request, and then synchronize the second synchronization information including the data call result. As a result, the data reconciliation processing node receives the second synchronization information synchronized by the target node within a preset time interval, including the data call result of the target node's response to the target service request.
[0128] Step 904: If the data call result includes: data call failure for target data at the target node, then the business processing result is determined to be business processing failure.
[0129] The data call result includes a failure to call the target data at the target node, or a success to call the target data at the target node. Based on this, if the data call result included in the second synchronization information includes a failure to call the target data at the target node, it means that the target node failed to call the target data based on the target business, that is, it means that the target node did not complete the target business. Therefore, the data reconciliation processing node determines that the business processing result is a business processing failure. At this time, there is a reconciliation relationship between the target node and the associated node, that is, the data reconciliation processing node needs to perform data reconciliation processing on the target data of the target node and the associated node.
[0130] On the contrary, if the data call result included in the second synchronization information includes that the data call of the target data at the target node is successful, the data reconciliation processing node determines that the business processing result is a business processing success.
[0131] In this embodiment, the data call results in the synchronization information are used to determine whether each node has successfully completed the call to the target data. Since the target business needs to call the target data, the completion result of the target business can be determined based on the data call result of the target data, thereby ensuring the accuracy of the determined business processing result.
[0132] Based on different business needs, calls to target data may include reading target data, writing to the target node, and deleting target data. Therefore, the judgment of data call results should also be determined based on the specific call method. The following will introduce in detail the data call results corresponding to each business need, as well as the method of determining business processing failure.
[0133] In one embodiment, Figure 11 As shown, in step 904, if the data call result includes: the data call of the target data at the target node fails, then determining the business processing result as a business processing failure includes:
[0134] Step 1002: If the data call result includes that the target data is not read from the target node, it is determined that the business processing result is a business processing failure.
[0135] Specifically, the target business request instructs the target data to be read from the target node and written to the associated node. Therefore, if the data call result indicates that the target data was not read from the target node, then the target data was not written to the associated node. Therefore, the target node did not complete the target business, and the data reconciliation processing node determines that the business processing result is a business processing failure.
[0136] For example, in a transfer scenario, a target service request instructs to transfer 1000 from a target node to an associated node. This means that 1000 must be read from the target node and written to the associated node. Therefore, if 1000 is not read from the target node, the associated node cannot write 1000 either, and the service processing fails.
[0137] In one embodiment, again referring to Figure 11 In step 904, if the data call result includes: a data call to the target data at the target node fails, determining that the business processing result is a business processing failure includes:
[0138] Step 1004: If the data call result includes that the target data is not written into the target node, it is determined that the business processing result is a business processing failure.
[0139] Specifically, a target service request instructs the target data to be read from the associated node and written to the target node. Therefore, if the data call result indicates that the target data was not written to the target node, there are two possible scenarios: the target data was read from the associated node but not written to the target node, or the target data was not written to the target node due to the associated node's failure to read the target data. In this case, the target node has not completed the target service, so the data reconciliation processing node determines the service processing result as a service processing failure.
[0140] For example, in a transfer scenario, a target service request instructs to transfer 1000 from an associated node to a target node. This means that 1000 must be read from the associated node and written to the target node. Therefore, if 1000 is not written to the target node, the service processing will be considered failed regardless of whether 1000 is read from the associated node.
[0141] It should be understood that the target business request can also be used to indicate the reading of target data from the associated node, and the reading of target data from the target data. Based on this, if the data call result includes that the target data is not written to the target node, regardless of whether the associated node reads the target data, the business processing result can be determined to be a business processing failure. For example, in the scenario of article recommendation, the target business request is used to indicate that text data T1 needs to be written to the associated node, and text data T1 needs to be written to the target node. Therefore, if the target node does not have text data T1, at this time, regardless of whether the associated node writes text data T1, the business processing is determined to have failed.
[0142] In one embodiment, again referring to Figure 11 In step 904, if the data call result includes: a data call to the target data at the target node fails, determining that the business processing result is a business processing failure includes:
[0143] Step 1006: If the data call result includes that the target data is not deleted from the target node, it is determined that the business processing result is a business processing failure.
[0144] Specifically, the target business request can also be used to instruct the deletion of target data from associated nodes, as well as the deletion of target data from target data. Based on this, if the data call result includes failure to write the target data to the target node, the data reconciliation processing node can determine that the business processing result is a business processing failure, regardless of whether the associated node reads the target data.
[0145] For example, Figure 12 As shown, in the article recommendation scenario, the article data 1102 published by the publisher will first be stored in the associated node 1104, and then pushed to the target node 1106 for storage through the associated node 1104. When the user uses the terminal to perform operations such as querying or browsing, the article data 1102 will be obtained from the storage of the target node 1106, and the article data 1102 and other article data will be displayed on the terminal display interface 1108. Based on this, if the publisher of the article data 1102 has deleted the article data 1102, the associated node 1104 has completed the deletion of the article data 1102, but the target node 1106 has not deleted the article data 1102, then the business processing result is determined to be business processing failure.
[0146] In this embodiment, the data call results in the synchronization information are specifically used to determine whether the target node completes call operations such as writing, reading, and deleting the target data based on different service requirements, thereby determining the completion results of the target business based on the results corresponding to different data call operations, further ensuring the accuracy of the determined business processing results.
[0147] In one embodiment, Figure 13 As shown, step 704, determining the service processing result corresponding to the target service request according to the second synchronization information, includes:
[0148] Step 1202: If the data call result corresponding to the target service request does not exist in the second synchronization information, it is determined that the service processing result corresponding to the target service request is a service processing failure.
[0149] It can be seen from the above embodiments that after the target node obtains the data call result corresponding to the target business request, it will synchronize the second synchronization information including the data call result. However, in actual applications, when the target node synchronizes the second synchronization information again, due to an abnormal problem, the target data has not yet been called, indicating that the target node has not yet completed the call operation indicated by the target business request. At this time, the data call result corresponding to the target business request does not exist in the second synchronization information. Therefore, if the data call result corresponding to the target business request does not exist in the synchronized second synchronization information, the data reconciliation processing node also determines that the business processing result is a business processing failure.
[0150] In this embodiment, special circumstances in actual applications are further taken into consideration. When there is no data call result in the second synchronization information, it is directly determined that the business processing has failed so as to perform reconciliation processing, thereby improving the specific practicality of this solution.
[0151] In one embodiment, Figure 14 As shown, the method further includes:
[0152] Step 1302: If the second synchronization information synchronized by the target node is not received within the preset time interval, it is determined that the service processing result is a service processing failure.
[0153] It can be seen from the above embodiments that after the target node receives the target service request, it will call the target data according to the target service request to process the target service, obtain the data call result corresponding to the target service request of the node, and then synchronize the second synchronization information including the data call result.
[0154] However, if the target node experiences an abnormality such as node downtime, the target node cannot call the target data based on the target service request, that is, it cannot complete the target service request, and therefore cannot synchronize the second synchronization information within the preset time interval. Secondly, if the target node encounters network environment problems or other abnormalities, even if the target node completes the call of the target data based on the target service request, the second synchronization information containing the data call result cannot be synchronized due to the network problem.
[0155] Therefore, the data reconciliation processing node does not receive the second synchronization information synchronized by the target node within the preset time interval. At this time, the business processing result is determined to be a business processing failure, that is, the target business request is judged to be equivalent to failure, and a reconciliation operation is required on the target data.
[0156] In this embodiment, further consideration is given to abnormal situations such as node downtime or network environment problems in actual applications. If the second synchronization information synchronized by the target node is not received within the preset time interval, it is determined that the business processing has failed and reconciliation processing is performed, thereby further improving the specific practicality of this solution.
[0157] In one embodiment, Figure 15 As shown, the method further includes:
[0158] Step 1402 : Counting the service processing results corresponding to the multiple target service requests within a preset time period.
[0159] The business processing results include both failed and successful processing. Based on this, the data reconciliation processing node collects statistics on the business processing results corresponding to each target business request within a preset time period. Specifically, the preset time period can be 1 day, 3 days, 7 days, or 15 days. The specific preset time period needs to be determined based on actual conditions and business needs, and this example should not be construed as limiting.
[0160] Step 1404 : Based on the business processing results corresponding to each target business request, the business processing results are calculated as data distribution of business processing failures and business processing successes.
[0161] Specifically, the data reconciliation processing node calculates the distribution relationship between the number of business processing failures and the preset time period, and calculates the distribution relationship between the number of business processing successes and the preset time period.
[0162] In step 1406, an evaluation score is calculated based on the data distribution of failed and successful business processing. The evaluation score is used to describe the proportion of data inconsistencies in business processing within a preset time period.
[0163] Among them, the evaluation score is used to describe the proportion of data inconsistencies in business processing within a preset time period, that is, the proportion of business processing failures in the business processing results. Specifically, the data reconciliation processing node calculates the percentage of business processing failures in the business processing results, and calculates the percentage of business processing successes in the business processing results, and determines the evaluation score based on this. It should be understood that in actual applications, based on the evaluation score, a business health score can be further obtained. The business health score shows the proportion of successful business processing within a preset time period, and the evaluation score and business health score and other data are displayed on the terminal.
[0164] For example, if the business processing results include 2 business processing failures and 98 business processing successes, the data distribution can include: business processing failures account for 2% (2 / 100) and business processing successes account for 98% (98 / 100). The evaluation score can then be 2%, and the business health score can be 98%.
[0165] For ease of understanding, Figure 16As shown, based on the data distribution 1502 of successful business processing and the data distribution 1504 of reconciliation based on failed business processing, an evaluation score 1506 of 2% can be obtained, indicating that 2% of business processing failures were reconciled within the preset time period, and further a business health score 1508 of 98% was obtained, indicating that 98% of business processing was successful within the preset time period.
[0166] In this embodiment, the data distribution of business processing failures and business processing successes is abstracted into a unified evaluation score and a business health score, thereby being able to more accurately describe the business processing situation within a preset time period.
[0167] The following is a detailed introduction to the specific process of data reconciliation processing, such as Figure 17 As shown, a data reconciliation processing method is provided, which is applied to Figure 4 Taking the data reconciliation processing node 402 in
[15] as an example, the following steps are included:
[0168] Step 1602: Receive first synchronization information synchronized by the target node.
[0169] The first synchronization information includes a target service request, which indicates the call to target data to process a target service. Specifically, the target service requires the target node and associated nodes to jointly call the target data to process the target service. Specifically, the data reconciliation processing node receives the first synchronization information in a manner similar to step 702 and is not further described here.
[0170] In this embodiment, there are three specific ways to determine that the business processing result is a business processing failure. The following introduces these three ways respectively in step 1604, step 1606, step 1608, and step 1610. It should be understood that the step number does not cause a time sequence limitation.
[0171] Step 1604: If the second synchronization information of the target node synchronization is received within the preset time interval, the data call result corresponding to the target service request included in the second synchronization information is obtained.
[0172] The "business processing result" specifically describes the result of each node processing the target business based on the target business request, and the "data call result" specifically describes the result of each node calling the target data based on the target business. Specifically, the specific method for the data reconciliation processing node to receive the second synchronization information is similar to step 704, and the specific method for obtaining the data call result is similar to step 902, which will not be repeated here.
[0173] Step 1606: If the data call result includes: data call failure for target data at the target node, then the business processing result is determined to be business processing failure.
[0174] The data call result includes a failure to call the target data at the target node, or a success to call the target data at the target node. Based on this, if the data call result included in the second synchronization information includes a failure to call the target data at the target node, it means that the target node failed to call the target data based on the target business, that is, it means that the target node did not complete the target business. Therefore, the data reconciliation processing node determines that the business processing result is a business processing failure. At this time, there is a reconciliation relationship between the target node and the associated node, that is, the data reconciliation processing node needs to perform data reconciliation processing on the target data of the target node and the associated node.
[0175] Step 1608: If the second synchronization information of the target node synchronization is received within the preset time interval, and the second synchronization information does not contain the data call result corresponding to the target service request, it is determined that the service processing result corresponding to the target service request is service processing failure.
[0176] In actual applications, when the target node synchronizes the second synchronization information again, due to an exception, the target data has not yet been called. This indicates that the target node has not yet completed the call operation indicated by the target business request. At this time, the data call result corresponding to the target business request does not exist in the second synchronization information. Therefore, if the synchronized second synchronization information does not contain the data call result corresponding to the target business request, the data reconciliation processing node also determines that the business processing result is a business processing failure.
[0177] Step 1610: If the second synchronization information synchronized by the target node is not received within the preset time interval, it is determined that the service processing result is a service processing failure.
[0178] If the target node experiences an abnormality such as node downtime, the target node is unable to retrieve the target data based on the target service request, meaning it cannot complete the target service request and, therefore, cannot synchronize the second synchronization information within the preset time interval. Secondly, if the target node encounters network environment issues or other abnormalities, even if the target node completes the target data call based on the target service request, the network issue will prevent it from synchronizing the second synchronization information containing the data call results.
[0179] Therefore, the data reconciliation processing node does not receive the second synchronization information synchronized by the target node within the preset time interval. At this time, the business processing result is determined to be a business processing failure, that is, the target business request is judged to be equivalent to failure, and a reconciliation operation is required on the target data.
[0180] Step 1612: Obtain the number of services running on each service node.
[0181] Among them, the number of businesses is the number of businesses that are run when the business service node performs data reconciliation processing on the target data of the target node and the associated nodes. Based on this, if the determined business processing result is a business processing failure, it means that the target node has failed to call the target data based on the target business, that is, it means that the target business has not been completed. At this time, the data reconciliation processing node can determine that data reconciliation processing is required for the target data. Based on this, in scenarios where there is a large amount of data to be reconciled, it is necessary to further consider how to balance the load of data reconciliation processing on each business service node. Therefore, before performing data reconciliation processing, the data reconciliation processing node needs to obtain the number of businesses that are run when each business service node performs data reconciliation processing on the target data of the target node and the associated nodes.
[0182] Step 1614 , based on the number of businesses running on each business service node, calculate the load rate of each business service node in performing data reconciliation processing on the target data of the target node and the associated nodes.
[0183] The load rate is specifically used to describe the load rate of the corresponding business service node when performing data reconciliation processing, that is, the ratio of the actual number of businesses when the business service node performs data reconciliation processing to the maximum number of businesses that the business service node can carry. Based on this, the data reconciliation processing node calculates the load rate of the business service node when performing data reconciliation processing on the target node and the associated nodes using the aforementioned formula (2), based on the number of businesses running on each business service node and the reconciliation business determined to perform data reconciliation processing on the target data of the target node and the associated nodes.
[0184] Step 1616 : Among the load rates of each business service node performing data reconciliation processing on the target data of the target node and the associated nodes, the business service node corresponding to the smallest load rate is determined as the target business service node.
[0185] The target business service node is the service node that performs data reconciliation processing on the target data of the target node and associated nodes. Specifically, the data reconciliation processing node sorts the load rates of each business service node performing data reconciliation processing on the target data of the target node and associated nodes from high to low based on the load rates, and determines the business service node with the lowest load rate as the target business service node.
[0186] Step 1618: Perform data reconciliation processing on the target data of the target node and the associated nodes through the target business service node.
[0187] Specifically, after the data reconciliation processing node determines the target business service node through step 1616, it will perform data reconciliation processing on the target data of the target node and the associated nodes through the target business service node. The method of performing data reconciliation processing is similar to step 706 and will not be repeated here.
[0188] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.
[0189] Based on the same inventive concept, embodiments of the present application also provide a data reconciliation processing device for implementing the aforementioned data reconciliation processing method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more data reconciliation processing device embodiments provided below can be found in the above-mentioned limitations of the data reconciliation processing method and will not be repeated here.
[0190] In one embodiment, Figure 18 As shown, a data reconciliation processing device is provided, including: a communication module 1702 and a processing module 1704, wherein:
[0191] The communication module 1702 is configured to receive first synchronization information synchronized by a target node, where the first synchronization information includes a target service request, where the target service request is used to instruct to call target data to process a target service;
[0192] Processing module 1704 is used to determine the business processing result corresponding to the target business request based on the second synchronization information if the communication module 1702 receives the second synchronization information synchronized by the target node within a preset time interval; if the business processing result is a business processing failure, perform data reconciliation processing on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node.
[0193] In one embodiment, the processing module 1704 is specifically used to obtain the number of businesses running on each business service node if the business processing result is a business processing failure; and based on the number of businesses running on each business service node, calculate the load rate of each business service node for data reconciliation processing of the target data of the target node and the associated nodes; and determine the business service node corresponding to the load rate with the smallest value as the target business service node; and perform data reconciliation processing on the target data of the target node and the associated nodes through the target business service node.
[0194] In one embodiment, the processing module 1704 is specifically used to obtain the data call result corresponding to the target business request included in the second synchronization information; if the data call result includes: the data call of the target data at the target node fails, then the business processing result is determined to be a business processing failure.
[0195] In one embodiment, the processing module 1704 is specifically configured to detect that a data call to target data at a target node fails, including: failing to read the target data from the target node.
[0196] In one embodiment, the processing module 1704 is specifically configured to fail to call the target data at the target node, including: failing to write the target data to the target node.
[0197] In one embodiment, the processing module 1704 is specifically configured to fail to call the target data at the target node, including: failing to delete the target data from the target node.
[0198] In one embodiment, the processing module 1704 is specifically configured to determine that the service processing result corresponding to the target service request is a service processing failure if the data call result corresponding to the target service request does not exist in the second synchronization information.
[0199] In one embodiment, the processing module 1704 is specifically configured to determine that the service processing result is a service processing failure if the communication module 1702 does not receive the second synchronization information synchronized by the target node within a preset time interval.
[0200] In one embodiment, the processing module 1704 is specifically used to count the business processing results corresponding to multiple target business requests within a preset time period; and based on the business processing results corresponding to each target business request, calculate the business processing results as the data distribution of business processing failures and business processing successes; and calculate the evaluation score through the data distribution of business processing failures and business processing successes. The evaluation score is used to describe the proportion of data inconsistencies in business processing within the preset time period.
[0201] Each module in the aforementioned data reconciliation processing device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.
[0202] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 19 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store target data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data reconciliation processing method is implemented.
[0203] Those skilled in the art will understand that Figure 19 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.
[0204] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.
[0205] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0206] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0207] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.
[0208] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.
[0209] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0210] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.
Claims
1. A data reconciliation processing method, characterized in that: The method comprises: Receiving first synchronization information synchronized by a target node, where the first synchronization information includes a target service request, where the target service request is used to instruct calling target data to process a target service; If second synchronization information synchronized by the target node is received within a preset time interval, determining a service processing result corresponding to the target service request according to the second synchronization information; If the business processing result is a business processing failure, performing data reconciliation processing on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node; If the business processing result is a business processing failure, performing data reconciliation processing on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node, including: If the business processing result is business processing failure, obtain the number of businesses running on each business service node; Calculating, based on the number of businesses running on each of the business service nodes, a load rate of each of the business service nodes performing data reconciliation processing on the target data of the target node and the associated nodes; The business service node corresponding to the smallest load rate is determined as the target business service node; The target business service node performs data reconciliation processing on the target data of the target node and the associated node.
2. The method according to claim 1, characterized in that The determining, according to the second synchronization information, a service processing result corresponding to the target service request includes: Obtaining a data call result corresponding to the target service request included in the second synchronization information; If the data call result includes: a data call failure on the target data at the target node, then the business processing result is determined to be the business processing failure.
3. The method according to claim 2, characterized in that The failure of calling the target data at the target node includes at least one of the following: Item 1: The target data is not read from the target node; Item 2: The target data is not written into the target node; Item 3: The target data is not deleted from the target node.
4. The method according to claim 1, wherein The determining, according to the second synchronization information, a service processing result corresponding to the target service request includes: If the data call result corresponding to the target service request does not exist in the second synchronization information, it is determined that the service processing result corresponding to the target service request is a service processing failure.
5. The method according to claim 1, wherein The method further comprises: If the second synchronization information synchronized by the target node is not received within the preset time interval, it is determined that the service processing result is a service processing failure.
6. The method according to claim 1, characterized in that The method further comprises: Count the business processing results corresponding to multiple target business requests within a preset time period; Based on the business processing results corresponding to each target business request, the business processing results are calculated as the data distribution of business processing failures and business processing successes; An evaluation score is calculated based on the data distribution of failed and successful business processing, and the evaluation score is used to describe the proportion of data inconsistencies in business processing within the preset time period.
7. A data reconciliation processing device, characterized in that: The device comprises: A communication module, configured to receive first synchronization information synchronized by a target node, wherein the first synchronization information includes a target service request, and the target service request is used to instruct to call target data to process a target service; a processing module configured to, if the communication module receives second synchronization information synchronized by the target node within a preset time interval, determine a business processing result corresponding to the target business request based on the second synchronization information; if the business processing result is a business processing failure, obtain the number of businesses running on each business service node; if the business processing result is a business processing failure, perform data reconciliation processing on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node; The processing module is specifically used to calculate the load rate of each business service node for performing data reconciliation processing on the target data of the target node and the associated nodes based on the number of businesses running on each business service node; determine the business service node corresponding to the smallest load rate as the target business service node; and perform data reconciliation processing on the target data of the target node and the associated nodes through the target business service node.
8. The device according to claim 7, characterized in that The processing module is specifically configured to obtain a data call result corresponding to the target service request included in the second synchronization information; If the data call result includes: a data call failure on the target data at the target node, then the business processing result is determined to be the business processing failure.
9. The device according to claim 8, characterized in that The failure of calling the target data at the target node includes at least one of the following: Item 1: The target data is not read from the target node; Item 2: The target data is not written into the target node; Item 3: The target data is not deleted from the target node.
10. The device according to claim 7, characterized in that The processing module is specifically configured to determine that the business processing result corresponding to the target business request is a business processing failure if the data call result corresponding to the target business request does not exist in the second synchronization information.
11. The device according to claim 7, characterized in that The processing module is specifically configured to determine that the service processing result is a service processing failure if the second synchronization information synchronized by the target node is not received within the preset time interval.
12. The device according to claim 7, characterized in that The processing module is specifically used to count the business processing results corresponding to multiple target business requests within a preset time period; based on the business processing results corresponding to each target business request, calculate the business processing results as the data distribution of business processing failures and business processing successes; through the data distribution of business processing failures and business processing successes, calculate an evaluation score, and the evaluation score is used to describe the proportion of data inconsistencies in business processing within the preset time period.
13. A data reconciliation processing device, characterized in that: The device comprises: A configuration management module is configured to provide add, delete, modify, and query logic for consistency scenarios and event pool configurations, including a consistency event pool and an error event pool. The consistency event pool is used to trigger the reception of first synchronization information for target node synchronization, wherein the first synchronization information includes a target service request, which is used to instruct the call of target data to process the target service. An event management module is configured to manage the consistency event pool; if second synchronization information synchronized by the target node is received within a preset time interval, determine a service processing result corresponding to the target service request according to the second synchronization information; an engine module configured to, if the business processing result is a business processing failure, obtain the number of businesses running on each business service node; and, if the business processing result is a business processing failure, perform data reconciliation processing on the target data of the target node and the associated nodes that have a reconciliation relationship with the target node; The engine module is specifically configured to calculate, based on the number of businesses running on each of the business service nodes, a load rate of each of the business service nodes for performing data reconciliation processing on the target data of the target node and the associated nodes; determine the business service node corresponding to the smallest load rate as the target business service node; and perform data reconciliation processing on the target data of the target node and the associated nodes through the target business service node; The error event management module is used to manage the error event pool.
14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Data reconciliation method and device, electronic equipment and machine readable storage medium
CN113377757A