Data reconciliation method, device and equipment across systems and storage medium
By using a multi-dimensional reconciliation method, the problem of rough and inaccurate results caused by traditional single-dimensional reconciliation is solved, and accurate verification and consistency of cross-system data storage tables are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU PINWEI SOFTWARE CO LTD
- Filing Date
- 2026-03-31
- Publication Date
- 2026-06-30
Smart Images
Figure CN122309569A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of cross-system data reconciliation technology, specifically to a cross-system data reconciliation method, apparatus, device, and storage medium. Background Technology
[0002] In today's highly digitalized business and financial environment, data interactions between different systems, platforms, or participants are becoming increasingly frequent and complex. Data reconciliation, as a key link to ensure data consistency, detect discrepancies, and safeguard business and financial accuracy, has become a core back-office process across all industries.
[0003] Traditional reconciliation processes mainly rely on single-dimensional comparisons. While this approach is direct and simple, it cannot fully meet reconciliation needs and often misses a lot of data that needs to be reconciled. As a result, the final reconciliation results are rather rough and inaccurate, affecting normal data interaction between systems. Summary of the Invention
[0004] In view of this, this application provides a cross-system data reconciliation method, apparatus, device, and storage medium to solve the problem that existing reconciliation processes cannot fully meet reconciliation requirements, often missing a lot of data that needs to be reconciled, resulting in a rough and inaccurate final reconciliation result that affects normal data interaction between systems.
[0005] To achieve the above objectives, the following solution is proposed: Firstly, a cross-system data reconciliation method includes: Obtain cross-system business reconciliation requirements; Identify the two systems to be reconciled corresponding to the business reconciliation requirements, and extract the reconciliation data storage tables for each of these two systems. Identify multiple reconciliation dimensions for these two reconciliation data storage tables; Each reconciliation rule is selected from a pre-set set of reconciliation rules based on each of the aforementioned reconciliation dimensions; Based on the reconciliation rules described above, the two reconciliation data storage tables are reconciled to obtain the target reconciliation result.
[0006] Preferably, determining the multiple reconciliation dimensions of these two reconciliation data storage tables includes: For data stored in one reconciliation data storage table, multiple first data dimensions are defined; at the same time, for data stored in another reconciliation data storage table, multiple second data dimensions are defined. Each of the first data dimensions is combined into a first data dimension set, and each of the second data dimensions is combined into a second data dimension set; Find the union of the first data dimension set and the second data dimension set; For each data dimension in the collection, a corresponding reconciliation dimension is set.
[0007] Preferably, the step of reconciling the two reconciliation data storage tables based on each of the reconciliation rules to obtain the target reconciliation result includes: For each of the reconciliation rules, if the reconciliation rule is a field reconciliation rule, then the field reconciliation rule is used to perform field reconciliation on the two reconciliation data storage tables to obtain the first reconciliation result; If the reconciliation rule is an aggregated reconciliation rule, then the two reconciliation data storage tables are aggregated and reconciled using the aggregated reconciliation rule to obtain the second reconciliation result; If the reconciliation rule is a JSON reconciliation rule, then the two reconciliation data storage tables are reconciled using the JSON reconciliation rule to obtain a third reconciliation result; The first reconciliation result, the second reconciliation result, and / or the third reconciliation result are combined to obtain the target reconciliation result.
[0008] Preferably, the step of performing field reconciliation on the two reconciliation data storage tables using the field reconciliation rules to obtain a first reconciliation result includes: Determine the first fields in one reconciliation data storage table, and simultaneously determine the second fields in another reconciliation data storage table; Each of the first fields is matched with each of the second fields to form field pairs; The reconciliation rules for each field are used to reconcile the accounts, resulting in the first reconciliation result.
[0009] Preferably, the step of aggregating and reconciling the two reconciliation data storage tables using the aggregated reconciliation rules to obtain a second reconciliation result includes: Aggregate the fields in one reconciliation data storage table to obtain the first aggregated values, and simultaneously aggregate the fields in another reconciliation data storage table to obtain the second aggregated values; Each of the first aggregated values is mapped to each of the second aggregated values to form aggregated value pairs; The aggregated reconciliation rules are used to reconcile each of the aggregated value pairs to obtain a second reconciliation result.
[0010] Preferably, the step of performing JSON reconciliation on the two reconciliation data storage tables using the JSON reconciliation rules to obtain a third reconciliation result includes: Identify each field of type JSON in one reconciliation data storage table as the first JSON field, and simultaneously identify each field of type JSON in another reconciliation data storage table as the second JSON field; The values of each of the first JSON fields are extracted using pre-defined expressions to obtain the respective first JSON values; The values of each of the second JSON fields are extracted using the expression to obtain the respective second JSON values; Each of the first JSON values is mapped to each of the second JSON values to form JSON value pairs; The JSON reconciliation rules are used to reconcile each of the JSON value pairs to obtain a third reconciliation result.
[0011] Preferably, after obtaining the reconciliation results, the method further includes: Identify any abnormal data in the target reconciliation results; For the target reconciliation results, multiple consecutive time periods are set; The number of abnormal reconciliations within each time period is determined from the abnormal data; Based on the number of abnormal reconciliations within each time period, an abnormal trend chart is created with time as the horizontal axis and the number of abnormal reconciliations as the vertical axis. Extract the amount of abnormal data for each field from the abnormal data; Establish a data distribution map based on the amount of abnormal data in each field; The entire reconciliation process is divided into progress segments to create a reconciliation progress dashboard. The system comprises a multi-dimensional chart consisting of the anomaly trend chart, the data distribution chart, and the reconciliation progress dashboard, and displays the multi-dimensional chart.
[0012] Secondly, a cross-system data reconciliation device includes: The business reconciliation requirement acquisition module is used to acquire cross-system business reconciliation requirements; The reconciliation data storage table extraction module is used to identify two systems to be reconciled corresponding to the business reconciliation requirements, and extract the reconciliation data storage table of each of the two systems to be reconciled. The reconciliation dimension determination module is used to determine one or more reconciliation dimensions for these two reconciliation data storage tables; The reconciliation rule selection module is used to select each reconciliation rule from a pre-set reconciliation rule set according to each of the reconciliation dimensions; The reconciliation module is used to reconcile the two reconciliation data storage tables based on the reconciliation rules to obtain the target reconciliation result.
[0013] Thirdly, a cross-system data reconciliation device, including a memory and a processor; The memory is used to store programs; The processor is configured to execute the program to implement the steps of the cross-system data reconciliation method as described in any of the first aspects.
[0014] Fourthly, a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the cross-system data reconciliation method as described in any of the first aspects.
[0015] As can be seen from the above technical solution, this application obtains cross-system business reconciliation requirements; identifies two systems to be reconciled corresponding to the business reconciliation requirements, and extracts the reconciliation data storage tables for each of these two systems; determines multiple reconciliation dimensions for these two reconciliation data storage tables; selects reconciliation rules from a pre-set set of reconciliation rules based on each of the reconciliation dimensions; and reconciles the two reconciliation data storage tables based on each of the reconciliation rules to obtain the target reconciliation result. This application first obtains cross-system business reconciliation requirements, thereby determining which two systems need to be reconciled from the business reconciliation requirements, and then extracts their respective reconciliation data storage tables. This application starts from multiple reconciliation dimensions, executes reconciliation processes under multiple reconciliation dimensions, and reconciles the two reconciliation data storage tables separately. The resulting target reconciliation result is comprehensive, detailed, and accurate, achieving precise verification of cross-system reconciliation data storage tables and ensuring consistency. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0017] Figure 1 An optional flowchart of a cross-system data reconciliation method provided in an embodiment of this application; Figure 2 An optional flowchart of another cross-system data reconciliation method provided in an embodiment of this application; Figure 3 A schematic diagram of the structure of a cross-system data reconciliation device provided in this application embodiment; Figure 4 This is a schematic diagram of a cross-system data reconciliation device provided in an embodiment of this application. Detailed Implementation
[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0019] In today's highly digitalized business and financial environment, data interactions between different systems, platforms, or participants are becoming increasingly frequent and complex. Data reconciliation, as a key link to ensure data consistency, detect discrepancies, and safeguard business and financial accuracy, has become a core back-office process across all industries.
[0020] Traditional reconciliation processes mainly rely on single-dimensional comparisons. While this approach is direct and simple, it cannot fully meet reconciliation needs and often misses a lot of data that needs to be reconciled. As a result, the final reconciliation results are rather rough and inaccurate, affecting normal data interaction between systems.
[0021] To address the shortcomings of the prior art, embodiments of the present invention provide a cross-system data reconciliation method. This method can be applied to various computer terminals or smart terminals, and its execution entity can be the processor or server of the computer terminal or smart terminal. The present invention can be used in numerous general-purpose or special-purpose computing device environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, and distributed computing environments including any of the above devices, etc.
[0022] The flowchart of the method is as follows: Figure 1 As shown, it specifically includes: S1: Obtain cross-system business reconciliation requirements.
[0023] Obtaining reconciliation business requirements defines the business context and scope of reconciliation. It transforms the vague data that needs to be reconciled into a clear technical object that can be processed by the system, thereby defining the goal of this reconciliation, such as which systems are being reconciled, and providing clear guidance for subsequent reconciliation processes.
[0024] Of course, in addition to determining which systems are being reconciled, it is also possible to determine what type of data is being reconciled, what time of data is being reconciled, and so on. This embodiment does not impose any restrictions on these aspects.
[0025] S2: Identify the two systems to be reconciled corresponding to the business reconciliation requirements, and extract the reconciliation data storage tables of each of the two systems to be reconciled.
[0026] From the business reconciliation requirements, two systems to be reconciled can be identified, such as system A and system B. In this application, system A has its own reconciliation data storage table, and system B also has its own corresponding reconciliation data storage table, which are bound together. This allows this application to focus on the data interface it provides without relying on the implementation details of the specific system.
[0027] Therefore, this step specifies that the data used for subsequent reconciliation comes from the reconciliation data storage tables of each system to be reconciled. Regardless of whether the underlying data storage table is a MySQL, Oracle, or Hive table, this application treats it as an input source with a unified format, thus shielding the heterogeneity of the data sources and laying the foundation for subsequent data comparison.
[0028] The reconciliation data storage table stores the data used by the corresponding reconciliation system for reconciliation. The data stored in the reconciliation data storage table can be selected arbitrarily according to different business reconciliation needs. This embodiment does not impose any restrictions on this.
[0029] S3: Determine multiple reconciliation dimensions for these two reconciliation data storage tables.
[0030] This application introduces a multi-dimensional reconciliation mechanism, which enables the consistency of reconciliation data to be verified from multiple perspectives, thereby improving the accuracy and depth of data reconciliation.
[0031] Different reconciliation dimensions can assume different reconciliation roles. In one example, the reconciliation dimensions include order number, transaction amount, transaction status, and user ID. The order number is used as the primary associated dimension, the transaction amount as the core verification dimension, and the transaction status as the auxiliary confirmation dimension. By reconciling from these dimensions separately, the problem of inaccurate reconciliation from a single dimension can be avoided, and more complex discrepancies can be identified, such as matching order numbers but inconsistent transaction amounts, or identical order numbers and transaction amounts but different transaction statuses.
[0032] S4: Select each reconciliation rule from the pre-set reconciliation rule set according to each of the reconciliation dimensions.
[0033] This application pre-sets a set of reconciliation rules, which includes various reconciliation rules. These rules can be pre-used rules, newly created rules, or a new set of reconciliation rules can be created based on business reconciliation needs and the systems to be reconciled. This allows for flexible reconciliation. If other reconciliation processes are to be performed later, only the set of reconciliation rules needs to be updated, or the reconciliation rules themselves need to be updated by making the corresponding configurations. This also allows for a better match with business reconciliation needs and makes the reconciliation results more accurate.
[0034] S5: Reconcile the two reconciliation data storage tables based on the reconciliation rules to obtain the target reconciliation result.
[0035] Once the reconciliation rules are determined, the two reconciliation data storage tables can be reconciled according to each reconciliation rule to obtain the final target reconciliation result. This target reconciliation result is not a log compiled from reconciliation data, but rather structured data that can be directly used for subsequent processing, such as displaying differences, generating reports, and facilitating subsequent result traceability.
[0036] As can be seen from the above technical solution, this application obtains cross-system business reconciliation requirements; identifies two systems to be reconciled corresponding to the business reconciliation requirements, and extracts the reconciliation data storage tables for each of these two systems; determines multiple reconciliation dimensions for these two reconciliation data storage tables; selects reconciliation rules from a pre-set set of reconciliation rules based on each of the reconciliation dimensions; and reconciles the two reconciliation data storage tables based on each of the reconciliation rules to obtain the target reconciliation result. This application first obtains cross-system business reconciliation requirements, thereby determining which two systems need to be reconciled from the business reconciliation requirements, and then extracts their respective reconciliation data storage tables. This application starts from multiple reconciliation dimensions, executes reconciliation processes under multiple reconciliation dimensions, and reconciles the two reconciliation data storage tables separately. The resulting target reconciliation result is comprehensive, detailed, and accurate, achieving precise verification of cross-system reconciliation data storage tables and ensuring consistency.
[0037] Furthermore, existing technologies present the following problem: For business reconciliation needs, a task is created in the data warehouse. This task handles three logics: 1) acquiring data sources from two systems; 2) cleaning and processing the data sources to obtain two intermediate tables; and 3) reconciling the two intermediate tables. This results in multiple processes within a single task. When the logic of this task needs to be modified, it is necessary to check whether the data processing logic of each of the two systems needs to be adjusted, which is cumbersome and wasteful of resources. In this application, the data of the two systems to be reconciled are stored separately, and tasks can be set for each system. Each task stores its own reconciliation data. Then, during reconciliation, a piece of Spark SQL code is automatically generated according to the reconciliation rules. Based on the Spark SQL code, a reconciliation task is automatically generated. This task is specifically used to handle the reconciliation process. Thus, each system to be reconciled only needs to maintain its own reconciliation data, thereby saving resources and simplifying the process. Furthermore, suppose there is another scenario that requires the use of the reconciliation data of another system. If the process of existing technology is followed, a new task needs to be written to obtain the reconciliation data of that system. However, this application can directly obtain the reconciliation data of that system. Moreover, by separating the maintenance of reconciliation data in each system and the specific reconciliation process, this application can improve the flexibility of reconciliation and prevent mutual interference.
[0038] The process for determining multiple reconciliation dimensions of the two reconciliation data storage tables in the method provided by this invention is described in detail below: For data stored in one reconciliation data storage table, multiple first data dimensions are defined; at the same time, for data stored in another reconciliation data storage table, multiple second data dimensions are defined. Each of the first data dimensions is combined into a first data dimension set, and each of the second data dimensions is combined into a second data dimension set; Find the union of the first data dimension set and the second data dimension set; For each data dimension in the collection, a corresponding reconciliation dimension is set.
[0039] Specifically, since different business reconciliation needs and different reconciliation systems require different data, the determination of reconciliation dimensions needs to be based on specific circumstances. That is, multiple data dimensions are set for the data stored in the two reconciliation data storage tables. Since the data dimensions are set for each reconciliation data storage table, there may be identical data dimensions in the first data dimension set and the second data dimension set, or there may be different data dimensions. Therefore, it is necessary to obtain a union to make the data dimensions more comprehensive. For example, the first data dimension set contains the first data dimensions: data dimension 1, data dimension 3, and data dimension 5, while the second data dimension set contains the second data dimensions: data dimension 2, data dimension 3, and data dimension 5. Then the data dimensions included in the union are: data dimension 1, data dimension 2, data dimension 3, and data dimension 5.
[0040] Next, for each data dimension in the collection, a corresponding reconciliation dimension can be set, that is, one data dimension corresponds to one reconciliation dimension.
[0041] In one example, if the data dimension is a field dimension, the corresponding reconciliation dimension is field reconciliation; if the data dimension is a field aggregation dimension, the corresponding reconciliation dimension is aggregation dimension; and if the data dimension is a JSON type field dimension, the corresponding reconciliation dimension is JSON dimension.
[0042] The following provides a detailed explanation of the process in this application of reconciling the two reconciliation data storage tables based on the various reconciliation rules to obtain the target reconciliation result. For details, please refer to [link / reference needed]. Figure 2 : S51: For each of the reconciliation rules, if the reconciliation rule is a field reconciliation rule, then the field reconciliation rule is used to perform field reconciliation on the two reconciliation data storage tables to obtain the first reconciliation result.
[0043] S52: If the reconciliation rule is an aggregated reconciliation rule, then the aggregated reconciliation rule is used to aggregate the two reconciliation data storage tables to obtain the second reconciliation result.
[0044] S53: If the reconciliation rule is a JSON reconciliation rule, then the two reconciliation data storage tables are reconciled using the JSON reconciliation rule to obtain a third reconciliation result.
[0045] S54: Combine the first reconciliation result, the second reconciliation result, and / or the third reconciliation result to obtain the target reconciliation result.
[0046] Specifically, this application pre-sets three reconciliation rules: field reconciliation rules, aggregate reconciliation rules, and JSON reconciliation rules. Each reconciliation rule corresponds to a reconciliation process. After determining the reconciliation rules corresponding to the business reconciliation requirements, it is necessary to reconcile the two reconciliation data storage tables according to each reconciliation rule to achieve the purpose of multi-dimensional reconciliation. The order is not important.
[0047] After obtaining the first, second, and third reconciliation results, they can be combined to form the target reconciliation result. During the combination process, considering that the first, second, and / or third reconciliation results are obtained from reconciliation of one, two, or three reconciliation dimensions respectively, there may be duplicate reconciliation results. Therefore, duplicates can be removed to obtain a concise and accurate target reconciliation result.
[0048] The following embodiments provide a detailed explanation of the steps in this application to perform field reconciliation on the two reconciliation data storage tables using the aforementioned field reconciliation rules to obtain the first reconciliation result.
[0049] Determine the first fields in one reconciliation data storage table, and simultaneously determine the second fields in another reconciliation data storage table; Each of the first fields is matched with each of the second fields to form field pairs; The reconciliation rules for each field are used to reconcile the accounts, resulting in the first reconciliation result.
[0050] Specifically, the field reconciliation rules apply to each field in the two reconciliation data storage tables. When reconciling one reconciliation data storage table with another, the fields in the two tables correspond to each other. Therefore, each field in one reconciliation data storage table can be matched with each field in the other reconciliation data storage table to form field pairs. If there is a single field that cannot form a field pair, it indicates that there may be a problem, and a special explanation can be made in the first reconciliation result.
[0051] You can reconcile the various field pairs generated using the field reconciliation rules. This allows you to check whether the two fields in a field pair are consistent or whether the two fields express the same meaning.
[0052] The following embodiments provide a detailed explanation of the steps in this application to aggregate and reconcile the two reconciliation data storage tables using the aggregated reconciliation rules to obtain a second reconciliation result.
[0053] Aggregate the fields in one reconciliation data storage table to obtain the first aggregated values, and simultaneously aggregate the fields in another reconciliation data storage table to obtain the second aggregated values; Each of the first aggregated values is mapped to each of the second aggregated values to form aggregated value pairs; The aggregated reconciliation rules are used to reconcile each of the aggregated value pairs to obtain a second reconciliation result.
[0054] Specifically, the prerequisite for using aggregated reconciliation rules is that the fields in the reconciliation data storage table need to be aggregated. During the aggregation process, not all fields are aggregated, but only those fields that can be aggregated are aggregated. For example, for a reconciliation data storage table, fields belonging to the same business are aggregated, and an aggregation function is selected to calculate the aggregated value. Then, a reconciliation data storage table may obtain one or more aggregated values. The same applies to another reconciliation data storage table. Similarly, according to the corresponding relationship, various aggregate pairs are formed, and reconciliation can be performed using aggregated reconciliation rules.
[0055] The following embodiments provide a detailed explanation of the steps in this application to perform JSON reconciliation on the two reconciliation data storage tables using the JSON reconciliation rules to obtain a third reconciliation result.
[0056] Identify each field of type JSON in one reconciliation data storage table as the first JSON field, and simultaneously identify each field of type JSON in another reconciliation data storage table as the second JSON field; The values of each of the first JSON fields are extracted using pre-defined expressions to obtain the respective first JSON values; The values of each of the second JSON fields are extracted using the expression to obtain the respective second JSON values; Each of the first JSON values is mapped to each of the second JSON values to form JSON value pairs; The JSON reconciliation rules are used to reconcile each of the JSON value pairs to obtain a third reconciliation result.
[0057] Specifically, the above process is a reconciliation process based on JSON reconciliation rules. First, the JSON fields in the reconciliation data storage table can be identified to determine which values to extract from the JSON fields to participate in the reconciliation. For example, a preset JSONPath expression can be used to point to the required data. Then, this data is converted into fields that the reconciliation process can understand as JSON values. Similarly, JSON value pairs are formed, and the JSON reconciliation rules are used to reconcile the JSON value pairs to obtain the third reconciliation result.
[0058] Optionally, after obtaining the reconciliation results, this application may also include: Identify any abnormal data in the target reconciliation results; For the target reconciliation results, multiple consecutive time periods are set; The number of abnormal reconciliations within each time period is determined from the abnormal data; Based on the number of abnormal reconciliations within each time period, an abnormal trend chart is created with time as the horizontal axis and the number of abnormal reconciliations as the vertical axis. Extract the amount of abnormal data for each field from the abnormal data; Establish a data distribution map based on the amount of abnormal data in each field; The entire reconciliation process is divided into progress segments to create a reconciliation progress dashboard. The system comprises a multi-dimensional chart consisting of the anomaly trend chart, the data distribution chart, and the reconciliation progress dashboard, and displays the multi-dimensional chart.
[0059] Specifically, to facilitate users' analysis and viewing of the target reconciliation results and improve their readability and analyzability, abnormal data can be identified and displayed. This requires analyzing the anomalies in the target reconciliation results from multiple perspectives and creating tables, which include three parts: an anomaly trend chart, a data distribution map, and a reconciliation progress dashboard.
[0060] The anomaly trend chart is designed to analyze the change in the number of abnormal reconciliations over time. Therefore, an anomaly trend chart can be created with time on the horizontal axis and the number of abnormal reconciliations on the vertical axis. This clearly shows the relationship between the number of abnormal reconciliations and time, and allows for analysis of changes in the number of abnormal reconciliations. The data distribution map analyzes from the perspective of fields. Since a field may contain abnormal data, the amount of abnormal data can be calculated. Then, the abnormal data amounts for each field are integrated to form a data distribution map. This allows for a unified view of the anomalies in each field, making it convenient for users to see the distribution of inconsistent data in each field. To help users understand the reconciliation progress and view the specific details of each reconciliation stage, a reconciliation progress dashboard can be created, which divides the entire reconciliation process into progress segments.
[0061] Finally, these three parts are combined to form a multi-dimensional chart, which is then displayed to users on the platform.
[0062] Furthermore, based on the presentation of the target reconciliation results in the aforementioned multi-dimensional charts, this application can also determine the data time of abnormal reconciliation data, as well as the main field data or unique field data corresponding to the abnormal reconciliation data. Then, it can analyze and examine the two reconciliation data storage tables separately to determine whether there is a problem with these two reconciliation data storage tables, or whether one of them has a problem.
[0063] If there are no problems with the two reconciliation data storage tables, it means that there was an error in the reconciliation process, and the reconciliation process can be re-executed to confirm again; if there is a problem with at least one reconciliation data storage table, it is necessary to trace back to the system to be reconciled for investigation and problem location.
[0064] This application allows you to set alarm priorities for abnormal reconciliation data, including high, medium, and low levels. The purpose is to help users quickly prioritize alarms that require immediate attention, ensuring that high-risk anomalies receive the first response. The alarm priority matching mode is as follows: 1) Sort by importance of business scenarios from highest to lowest.
[0065] 2) The alarm mechanism inherits the reconciliation priority by default. If there is a higher priority in the reconciliation process, the corresponding alarm will also have a higher priority level. This indicates that it is a high-risk alarm and users need to pay attention to it.
[0066] 3) Users can adjust the alarm priority according to their own situation, instead of using the default alarm priority rules.
[0067] Furthermore, this application can also simultaneously provide full-process management functions for anomaly data marking, root cause analysis, handling operations, and result verification, wherein: During the data tagging process, there is a data tagging button in the multi-dimensional charts displayed on the platform, which allows users to mark whether the data is abnormal. Since some abnormal reconciliation data is due to the time delay of the data itself, this part of the abnormal reconciliation data is actually normal business data, so users can tag it for easy classification later.
[0068] During the root cause analysis process, a root cause analysis button can be provided in the multi-dimensional charts displayed on the platform. After clicking, users can go directly to the corresponding reconciliation data storage table page to view the most original data source, which makes it convenient for users to confirm whether there are any real data reconciliation anomalies.
[0069] During the handling process, after the user confirms that a problem does exist, a work order needs to be created on the platform, and a person needs to be assigned to follow up and handle the matter.
[0070] During the result review process, the assigned person will locate and investigate the system to be reconciled corresponding to the abnormal reconciliation data after receiving feedback on the issue, and finally record the investigation results on the platform.
[0071] and Figure 1 Corresponding to the method described above, embodiments of the present invention also provide a cross-system data reconciliation device for reconciling data between systems. Figure 1In a specific implementation of the method, the cross-system data reconciliation device provided in this embodiment of the invention can be used on computer terminals or various mobile devices, combined with... Figure 3 This section introduces cross-system data reconciliation devices, such as... Figure 3 As shown, the device may include: The Business Reconciliation Requirement Acquisition Module 10 is used to acquire cross-system business reconciliation requirements. The reconciliation data storage table extraction module 20 is used to identify two systems to be reconciled corresponding to the business reconciliation requirements, and extract the reconciliation data storage table of each of the two systems to be reconciled. The reconciliation dimension determination module 30 is used to determine one or more reconciliation dimensions for these two reconciliation data storage tables; The reconciliation rule selection module 40 is used to select each reconciliation rule from a pre-set reconciliation rule set according to each of the reconciliation dimensions; The reconciliation module 50 is used to reconcile the two reconciliation data storage tables based on the reconciliation rules to obtain the target reconciliation result.
[0072] As can be seen from the above technical solution, this application obtains cross-system business reconciliation requirements; identifies two systems to be reconciled corresponding to the business reconciliation requirements, and extracts the reconciliation data storage tables for each of these two systems; determines multiple reconciliation dimensions for these two reconciliation data storage tables; selects reconciliation rules from a pre-set set of reconciliation rules based on each of the reconciliation dimensions; and reconciles the two reconciliation data storage tables based on each of the reconciliation rules to obtain the target reconciliation result. This application first obtains cross-system business reconciliation requirements, thereby determining which two systems need to be reconciled from the business reconciliation requirements, and then extracts their respective reconciliation data storage tables. This application starts from multiple reconciliation dimensions, executes reconciliation processes under multiple reconciliation dimensions, and reconciles the two reconciliation data storage tables separately. The resulting target reconciliation result is comprehensive, detailed, and accurate, achieving precise verification of cross-system reconciliation data storage tables and ensuring consistency.
[0073] Furthermore, embodiments of this application provide a cross-system data reconciliation device. Optionally, Figure 4 The hardware structure block diagram of the cross-system data reconciliation device is shown, with reference to Figure 4 The hardware structure of a cross-system data reconciliation device may include: at least one processor 01, at least one communication interface 02, at least one memory 03, and at least one communication bus 04.
[0074] In this embodiment, the number of processor 01, communication interface 02, memory 03 and communication bus 04 is at least one, and processor 01, communication interface 02 and memory 03 communicate with each other through communication bus 04.
[0075] Processor 01 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.
[0076] Memory 03 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device.
[0077] The memory stores a program that the processor can call. The program is used to execute the following cross-system data reconciliation methods, including: Obtain cross-system business reconciliation requirements; Identify the two systems to be reconciled corresponding to the business reconciliation requirements, and extract the reconciliation data storage tables for each of these two systems. Identify multiple reconciliation dimensions for these two reconciliation data storage tables; Each reconciliation rule is selected from a pre-set set of reconciliation rules based on each of the aforementioned reconciliation dimensions; Based on the reconciliation rules described above, the two reconciliation data storage tables are reconciled to obtain the target reconciliation result.
[0078] Optionally, the refined and extended functions of the program can be found in the description of the cross-system data reconciliation method in the method embodiments.
[0079] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor. When the program runs, it controls the device where the storage medium is located to perform the following cross-system data reconciliation method, including: Obtain cross-system business reconciliation requirements; Identify the two systems to be reconciled corresponding to the business reconciliation requirements, and extract the reconciliation data storage tables for each of these two systems. Identify multiple reconciliation dimensions for these two reconciliation data storage tables; Each reconciliation rule is selected from a pre-set set of reconciliation rules based on each of the aforementioned reconciliation dimensions; Based on the reconciliation rules described above, the two reconciliation data storage tables are reconciled to obtain the target reconciliation result.
[0080] Specifically, the storage medium can be a computer-readable storage medium, which can be an electronic storage device such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM.
[0081] Optionally, the refined and extended functions of the program can be found in the description of the cross-system data reconciliation method in the method embodiments.
[0082] Furthermore, the functional modules in the various embodiments of this disclosure can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the function is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a live streaming device, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this disclosure.
[0083] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0084] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0085] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A cross-system data reconciliation method, characterized in that, include: Obtain cross-system business reconciliation requirements; Identify the two systems to be reconciled corresponding to the business reconciliation requirements, and extract the reconciliation data storage tables for each of these two systems. Identify multiple reconciliation dimensions for these two reconciliation data storage tables; Each reconciliation rule is selected from a pre-set set of reconciliation rules based on each of the aforementioned reconciliation dimensions; Based on the reconciliation rules described above, the two reconciliation data storage tables are reconciled to obtain the target reconciliation result.
2. The method according to claim 1, characterized in that, The determination of the multiple reconciliation dimensions of these two reconciliation data storage tables includes: For data stored in one reconciliation data storage table, multiple first data dimensions are defined; at the same time, for data stored in another reconciliation data storage table, multiple second data dimensions are defined. Each of the first data dimensions is combined into a first data dimension set, and each of the second data dimensions is combined into a second data dimension set; Find the union of the first data dimension set and the second data dimension set; For each data dimension in the collection, a corresponding reconciliation dimension is set.
3. The method according to claim 1, characterized in that, The reconciliation of the two reconciliation data storage tables based on each of the reconciliation rules to obtain the target reconciliation result includes: For each of the reconciliation rules, if the reconciliation rule is a field reconciliation rule, then the field reconciliation rule is used to perform field reconciliation on the two reconciliation data storage tables to obtain the first reconciliation result; If the reconciliation rule is an aggregated reconciliation rule, then the two reconciliation data storage tables are aggregated and reconciled using the aggregated reconciliation rule to obtain the second reconciliation result; If the reconciliation rule is a JSON reconciliation rule, then the two reconciliation data storage tables are reconciled using the JSON reconciliation rule to obtain a third reconciliation result; The first reconciliation result, the second reconciliation result, and / or the third reconciliation result are combined to obtain the target reconciliation result.
4. The method according to claim 3, characterized in that, The process of reconciling the two reconciliation data storage tables using the aforementioned field reconciliation rules to obtain the first reconciliation result includes: Determine the first fields in one reconciliation data storage table, and simultaneously determine the second fields in another reconciliation data storage table; Each of the first fields is matched with each of the second fields to form field pairs; The reconciliation rules for each field are used to reconcile the accounts, resulting in the first reconciliation result.
5. The method according to claim 3, characterized in that, The process of aggregating and reconciling the two reconciliation data storage tables using the aggregated reconciliation rules to obtain a second reconciliation result includes: Aggregate the fields in one reconciliation data storage table to obtain the first aggregated values, and simultaneously aggregate the fields in another reconciliation data storage table to obtain the second aggregated values; Each of the first aggregated values is mapped to each of the second aggregated values to form aggregated value pairs; The aggregated reconciliation rules are used to reconcile each of the aggregated value pairs to obtain a second reconciliation result.
6. The method according to claim 3, characterized in that, The process of performing JSON reconciliation on the two reconciliation data storage tables using the aforementioned JSON reconciliation rules to obtain a third reconciliation result includes: Identify each field of type JSON in one reconciliation data storage table as the first JSON field, and simultaneously identify each field of type JSON in another reconciliation data storage table as the second JSON field; The values of each of the first JSON fields are extracted using pre-defined expressions to obtain the respective first JSON values; The values of each of the second JSON fields are extracted using the expression to obtain the respective second JSON values; Each of the first JSON values is mapped to each of the second JSON values to form JSON value pairs; The JSON reconciliation rules are used to reconcile each of the JSON value pairs to obtain a third reconciliation result.
7. The method according to any one of claims 1 to 6, characterized in that, After receiving the reconciliation results, the following is also included: Identify any abnormal data in the target reconciliation results; For the target reconciliation results, multiple consecutive time periods are set; The number of abnormal reconciliations within each time period is determined from the abnormal data; Based on the number of abnormal reconciliations within each time period, an abnormal trend chart is created with time as the horizontal axis and the number of abnormal reconciliations as the vertical axis. Extract the amount of abnormal data for each field from the abnormal data; Establish a data distribution map based on the amount of abnormal data in each field; The entire reconciliation process is divided into progress segments to create a reconciliation progress dashboard. The system comprises a multi-dimensional chart consisting of the anomaly trend chart, the data distribution chart, and the reconciliation progress dashboard, and displays the multi-dimensional chart.
8. A cross-system data reconciliation device, characterized in that, include: The business reconciliation requirement acquisition module is used to acquire cross-system business reconciliation requirements; The reconciliation data storage table extraction module is used to identify two systems to be reconciled corresponding to the business reconciliation requirements, and extract the reconciliation data storage table of each of the two systems to be reconciled. The reconciliation dimension determination module is used to determine one or more reconciliation dimensions for these two reconciliation data storage tables; The reconciliation rule selection module is used to select each reconciliation rule from a pre-set reconciliation rule set according to each of the reconciliation dimensions; The reconciliation module is used to reconcile the two reconciliation data storage tables based on the reconciliation rules to obtain the target reconciliation result.
9. A cross-system data reconciliation device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the various steps of the cross-system data reconciliation method as described in any one of claims 1-7.
10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the cross-system data reconciliation method as described in any one of claims 1-7.