Data verification method and device based on reconstructed system, equipment and storage medium

Abstract

The application discloses a data verification method and device based on a reconstruction system, equipment and a storage medium, and relates to the technical field of data processing. The method comprises the following steps: acquiring a first business data table in a first business system, and acquiring a second business data table matched with the first business data table from a second business system, wherein the first business system is a reconstructed business system, and the second business system is a business system before reconstruction; determining common table fields of the first business data table and the second business data table, and determining a comparison result of the first business data table and the second business data table according to field values of the common table fields; determining whether there is a difference field in the common table fields according to the comparison result; and if there is no difference field, determining that the first business system passes the data verification. The application can improve the accuracy of data verification.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data verification method, apparatus, device and storage medium based on a reconstruction system. Background Technology

[0002] As new technologies emerge and mature, companies often refactor their existing business systems based on new platforms or frameworks to improve their performance, security, and scalability. This can be done by migrating traditional host applications to the cloud, thus adapting to the needs of technological development.

[0003] After system refactoring, extensive testing is typically required to ensure consistent system functionality. However, relying solely on system testing without sufficient data verification and other auxiliary validation measures can lead to system defects caused by inadequate testing during actual operation. Therefore, refactored systems often attempt to introduce a "data verification" mechanism to improve verification quality. However, excessive reliance on manual verification results in low efficiency and a high risk of errors, especially when dealing with large datasets, making it difficult to guarantee comprehensive coverage of all verification points and potentially missing critical verification steps.

[0004] Therefore, improving the accuracy of data validation in the reconstruction system is a problem that urgently needs to be solved.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this application is to provide a data verification method, apparatus, device, and storage medium based on a reconstruction system, aiming to solve the technical problem of how to improve the accuracy of data verification in a reconstruction system.

[0007] To achieve the above objectives, this application proposes a data verification method based on a reconstructed system, applied to a first business system. The data verification method based on the reconstructed system includes:

[0008] Obtain a first business data table from a first business system, and obtain a second business data table from a second business system that matches the first business data table, wherein the first business system is the reconstructed business system, and the second business system is the business system before reconstruction;

[0009] Determine the common table fields of the first business data table and the second business data table, and determine the comparison result between the first business data table and the second business data table based on the field values ​​of each common table field;

[0010] Based on the comparison results, determine whether there are any differing fields in the common table fields;

[0011] If the difference field does not exist, then the first business system is determined to have passed data verification.

[0012] In one embodiment, the step of obtaining a second business data table that matches the first business data table from the second business system includes:

[0013] A data segmentation request is sent to the second business system. The data segmentation request includes the first business data table and data segmentation rules. The second business system receives the data segmentation request, determines a second business data table that matches the first business data based on the first business data table in the data segmentation request, segments the second business data table based on the data segmentation rules, determines multiple segmented data, and generates an index list based on the unique index of the first data entry in each segmented data. The index list is then sent to the first business system.

[0014] The system receives an index list sent by the second business system, generates multiple data acquisition requests based on multiple unique indexes in the index list, and sends each data acquisition request to the second business system. The second business system receives each data acquisition request, acquires segmented data corresponding to each unique index based on each data acquisition request and the unique index corresponding to the data acquisition request, and sends each segmented data to the first business system.

[0015] Receive multiple segments of data sent by the second service system, and merge the segments to obtain a second service data table.

[0016] In one embodiment, the step of sending a data segmentation request to the second business system includes:

[0017] The number of data entries for each segment of data is determined based on the first business data table, and the number of data entries for each segment of data is used as the data segmentation rule.

[0018] A data segmentation request is generated based on the data segmentation rules and the first business data.

[0019] The data segmentation request is sent to the second business system. The second business system receives the data segmentation request, matches a target data table in the second business system according to the first business data table in the data segmentation request, filters the corresponding data entries in the first business data table in the target data table, determines the second business data table based on all the filtered data entries, divides the second business data table into multiple segments according to the number of data entries in the data segmentation rules, generates an index list based on the unique index of the first data entry in each segment, and sends the index list to the first business system.

[0020] In one embodiment, after the step of merging the segmented data to obtain the second business data table, the method further includes:

[0021] Identify the table structure differences between the first business data table and the second business data table, wherein the table structure differences include differences in field names and differences in field types;

[0022] Based on the table structure of the first business data table, the field names and field types of the second business data table with different table structures are converted, and the second business data table is updated based on the converted field names and field types.

[0023] In one embodiment, after the step of merging the segmented data to obtain the second business data table, the method further includes:

[0024] Detect whether there are long fields in each field of the second business data table, wherein the long field is a field containing composite information or multiple attributes;

[0025] If the long field exists, determine the field content of the long field, and determine the splitting rule of the long field by parsing the field content;

[0026] The long field is split into multiple short fields according to the splitting rules, and each short field is named to obtain the field name of each short field;

[0027] In the second business data table, create multiple new fields corresponding to the short field, and fill the corresponding new fields with the field name and field content of each short field;

[0028] Update the second business data table according to the field name and field content of the new field.

[0029] In one embodiment, the step of determining that the first business system has passed data verification further includes:

[0030] Determine the number of fields in the common table fields of the first business data table and the second business data table;

[0031] The field values ​​corresponding to the common table fields in the first business data table are used as the first field values, and the field values ​​corresponding to the common table fields in the second business data table are used as the second field values.

[0032] Within a preset comparison time, obtain the number of times each of the first field values ​​and the corresponding second field values ​​are consistent.

[0033] Whether the system switching conditions are met is determined based on the number of fields in the common table and the number of times each first field value matches the corresponding second field value within the preset comparison time.

[0034] If the number of fields in the common table and the number of times each first field value matches the corresponding second field value within the preset comparison time are the same, then the system switching conditions are met, and the first business system is determined to have passed data verification.

[0035] In one embodiment, after determining that the first business system has passed data verification, the method further includes:

[0036] The business traffic destined for the second business system will be gradually switched to the first business system until all business traffic is directed to the first business system;

[0037] Gradually shut down the system services of the second business system.

[0038] Furthermore, to achieve the above objectives, this application also proposes a data verification device based on a reconstruction system, the data verification device based on the reconstruction system comprising:

[0039] The acquisition module is used to acquire a first business data table from a first business system and acquire a second business data table from a second business system that matches the first business data table, wherein the first business system is the reconstructed business system and the second business system is the business system before reconstruction;

[0040] The comparison module is used to determine the common table fields of the first business data table and the second business data table, and to determine the comparison result of the first business data table and the second business data table based on the field values ​​of each common table field.

[0041] The verification module is used to determine whether there are any differing fields in the common table fields based on the comparison results.

[0042] The verification module is used to determine that the first business system passes data verification if the difference field does not exist.

[0043] In addition, to achieve the above objectives, this application also proposes a data verification device based on a reconstruction system, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the data verification method based on the reconstruction system as described above.

[0044] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the data verification method based on the reconstructed system described above.

[0045] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the data verification method based on the reconstructed system described above.

[0046] This application proposes a data verification method based on a reconstructed system. The method involves obtaining a first business data table from a first business system and a second business data table from a second business system that matches the first business data table. The first business system is the reconstructed business system, and the second business system is the pre-reconstruction business system. Common table fields between the first and second business data tables are identified, and a comparison result is determined based on the values ​​of these common table fields. The comparison result is then used to determine if any discrepancies exist among the common table fields. If no discrepancies exist, the first business system passes data verification. This application utilizes automated methods for data comparison, ensuring data consistency between the reconstructed and pre-reconstruction business systems. Because automation can quickly process large amounts of data, it reduces the verification time and, compared to traditional manual verification methods, avoids potential human errors and improves the accuracy of data verification. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating an embodiment of the data verification method based on a reconstructed system provided in this application.

[0050] Figure 2 This is a schematic diagram of data segment loading and transmission provided in Embodiment 1 of the data verification method based on the reconstructed system of this application;

[0051] Figure 3 This is an example diagram of data segment loading and transmission provided in Embodiment 1 of the data verification method based on the reconstructed system of this application;

[0052] Figure 4 This is a schematic diagram of the table field-level comparison file structure provided in Embodiment 1 of the data verification method based on the reconstructed system of this application;

[0053] Figure 5 This is a schematic diagram of the table structure for the table field difference prompt configuration provided in Embodiment 1 of the data verification method based on the reconstructed system in this application;

[0054] Figure 6 This is an illustration of the table field scoring data provided in Embodiment 1 of the data verification method based on the reconstructed system in this application;

[0055] Figure 7 This is a flowchart of the table field scoring table update process provided in Embodiment 1 of the data verification method based on the reconstructed system in this application;

[0056] Figure 8 This is a diagram illustrating the reconstruction-verification-production relationship provided in Embodiment 1 of the data verification method based on the reconstruction system in this application.

[0057] Figure 9 A simplified flowchart illustrating the data verification method based on a reconstructed system provided in Embodiment 1 of this application;

[0058] Figure 10 This is a system switching maturity curve provided in Embodiment 2 of the data verification method based on the reconstructed system in this application;

[0059] Figure 11 This is a schematic diagram of the module structure of the data verification device based on the reconstruction system in an embodiment of this application;

[0060] Figure 12 This is a schematic diagram of the device structure of the hardware operating environment involved in the data verification method based on the reconstruction system in the embodiments of this application.

[0061] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0062] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0063] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0064] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or a first business system capable of performing the above functions. The following description uses the first business system as an example to illustrate this embodiment and the subsequent embodiments.

[0065] Based on this, embodiments of this application provide a data verification method based on a reconstructed system, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the data verification method based on the reconstructed system in this application.

[0066] In this embodiment, the data verification method based on the reconstructed system is applied to the first business system, and the data verification method based on the reconstructed system includes steps S10 to S40:

[0067] Step S10: Obtain the first business data table from the first business system, and obtain the second business data table that matches the first business data table from the second business system, wherein the first business system is the reconstructed business system, and the second business system is the business system before reconstruction.

[0068] It should be noted that the first business system refers to the restructured business system, and the second business system refers to the original business system. In the first business system, one or more business data tables are selected as the data to be verified. In the second business system, data with a similar structure to the first business data tables, containing the same field names (table fields), data types, and data ranges, is obtained.

[0069] Optionally, when selecting the first business data table, it is necessary to consider the importance of the data in the table and its impact on the business processes in the first business system. Typically, a data table containing key business logic is selected for verification. Since this embodiment applies to the first business system, it is necessary to interact with the second business system via network transmission to obtain the second business data table.

[0070] Understandably, obtaining the business data tables of the business systems before and after the restructuring provides a data foundation for subsequent data consistency verification.

[0071] In one feasible implementation, step S10 may include steps S11 to S13:

[0072] Step S11: Send a data segmentation request to the second business system. The data segmentation request includes a first business data table and data segmentation rules. The second business system receives the data segmentation request, determines a second business data table that matches the first business data based on the first business data table in the data segmentation request, segments the second business data table based on the data segmentation rules, determines multiple segmented data, and generates an index list based on the unique index of the first data entry in each segmented data. The index list is then sent to the first business system.

[0073] It should be noted that the initiator is the first business system, and the receiver is the second business system. The data segmentation request consists of two main parts: the first business data table, which contains the data set that needs to be processed in the first business system, and the data segmentation rules, which define how to divide the data into different segments. In the second business system, the data segmentation request is received from the first business system. Based on the content of the first business data table, the corresponding data table (the second business data table) is located within it. The second business data table is then segmented according to the data segmentation rules, resulting in multiple segments. For each segment, the unique index of the first data entry is extracted, and an index list containing the unique indexes of the first record of all segments is output. This index list is then sent back to the first business system.

[0074] In one feasible implementation, step S11 may include steps S111 to S113:

[0075] Step S111: Determine the number of data entries for each segment of data based on the first business data table, and use the number of data entries for each segment of data as the data segmentation rule.

[0076] It should be noted that, to process and transmit large amounts of data more efficiently, the data is usually divided into smaller blocks (segments). This helps reduce network load, improve data processing speed, and simplify error recovery processes. By analyzing the first business data table, we understand its structure, data volume, and other information, considering factors including but not limited to the table size, the complexity of data entries, available resources (such as bandwidth and storage), and performance requirements. Based on the results of the above analysis, we determine a suitable number of data entries that should be included in each segment. This number may be fixed, such as 1000 data records per segment; or it may be dynamic, determined based on certain algorithms or strategies. The determined number of data entries is incorporated into the data segmentation rules.

[0077] Alternatively, if the data volume is stable or does not require particularly complex logic to determine the segment size, a fixed value can be chosen as the number of data entries. For example, if the data table contains 10,000 records, each segment can contain 1,000 records, resulting in 10 segments for the entire dataset. For cases with significant data volume variations, a dynamic approach to determining the number of data entries can be used, taking into account factors such as data distribution characteristics, network conditions, and server capabilities. The size of the data segments affects the speed and efficiency of data processing; segments that are too small may lead to excessive communication overhead, while segments that are too large may result in excessively long processing times for individual segments.

[0078] Step S112: Generate a data segmentation request based on the data segmentation rules and the first business data;

[0079] It should be noted that, based on the data segmentation rules and the specific content of the first business data, a series of data segmentation requests are created to guide the subsequent data processing and distribution process. The data segmentation rules defined in step S111 specify the number of data entries that each segment should contain. Based on the content of the first business data, the data range that needs to be segmented is identified, and specific data segmentation requests are constructed according to the identified data range and the data segmentation rules.

[0080] Understandably, this step ensures that the data can be correctly divided into multiple segments according to predetermined rules. Through data segmentation requests, large-scale datasets can be effectively managed and processed while maintaining the efficiency and accuracy of data processing.

[0081] Step S113: The data segmentation request is sent to the second business system. The second business system receives the data segmentation request, matches the target data table in the second business system according to the first business data table in the data segmentation request, filters the corresponding data entries in the first business data table in the target data table, determines the second business data table based on all the filtered data entries, and divides the second business data table into multiple segments according to the number of data entries in the data segmentation rules. An index list is generated based on the unique index of the first data entry in each segment, and the index list is sent to the first business system.

[0082] It should be noted that after receiving the data segmentation request, the second business system needs to process the data according to the information in the request, find the corresponding entries, segment the data according to the data segmentation rules, and finally generate an index list to return to the first business system. The second business system receives the data segmentation request from the first business system and, based on the field names of the first business data table specified in the data segmentation request, searches for target data tables with the same field names in the second business system. For example, if the first business data table has a field named "Order ID", the second business system will search for tables containing the same field name. In the found target data tables, it filters out the entries that correspond to those in the first business data table. Based on all the filtered data entries, a new second business data table is formed, containing all the relevant data entries extracted from the target data table. According to the data segmentation rules (e.g., how many data entries each segment contains), the second business data table is divided into multiple segments. For each segment, the unique index of the first data entry is selected, and a list containing all these unique indexes is generated. Finally, the generated index list is sent back to the first business system.

[0083] This implementation can be applied to situations where one system requests specific data segments from another system, especially when the data volume is large and needs to be segmented according to specific rules. The first business system can effectively request the required data segments from the second business system and ensure data consistency and integrity.

[0084] Step S12: Receive the index list sent by the second business system, generate multiple data acquisition requests based on multiple unique indexes in the index list, and send each data acquisition request to the second business system. The second business system receives each data acquisition request, acquires the segmented data corresponding to each unique index based on each data acquisition request and the unique index corresponding to the data acquisition request, and sends each segmented data to the first business system.

[0085] It should be noted that the first business system receives an index list sent by the second business system. This index list contains multiple unique indexes, each corresponding to the first record of a data segment. Based on each unique index in the index list, a corresponding data retrieval request is generated. Each data retrieval request contains a unique index indicating which data segment the second business system needs to return. All data retrieval requests are sent to the second business system in parallel. The second business system receives the data retrieval requests from the first business system and uses the unique indexes in the requests to find and retrieve the corresponding data segments. For each data retrieval request, the second business system needs to search its stored data segments for data that matches the unique index in the request and then sends the retrieved data segments back to the first business system.

[0086] Step S13: Receive multiple segments of data sent by the second business system, and merge the segments to obtain the second business data table.

[0087] It should be noted that the first business system receives multiple segments of data sent from the second business system. These segments are obtained according to the data acquisition request generated in step S12, and each segment corresponds to a unique index. Data integrity is ensured and the received segment data is verified by comparing the number of received segments with the expected number and checking the correctness of the unique index of each segment. If the segmented data is arranged in a certain order, it needs to be sorted according to the unique index of the segmented data to ensure that the merged data table maintains its original order. The segmented data is then concatenated to form a continuous data set, and all the segmented data is merged into a complete data table. This complete data table contains information about all the segmented data received from the second business system, i.e., the complete second business data table.

[0088] Understandably, the first business system can integrate multiple segments of data received from the second business system into a complete second business data table. This not only ensures the integrity and accuracy of the data but also improves the efficiency of data transmission.

[0089] In this embodiment, a data segmentation transmission and loading method is proposed, which enables efficient batch transmission of data between different systems.

[0090] For example, Figure 2 A schematic diagram of data segmentation loading and transmission is provided, specifically:

[0091] The "segmented data loading" transfer function is suitable for any large-scale data transfer process and for data processing of different table designs.

[0092] First, the first business system requests data filtering and segmentation from the second business system's first business data table. The second business system then calculates the primary key column of the first record in each segment and returns the results.

[0093] Then, the first business system, in a multi-concurrency manner, requests detailed data from the second business system according to the primary key column of each segment. After obtaining the data, unnecessary fields are removed, different field names and types are converted, and large fields are further refined and split to obtain a dataset in a standard data format, which is then persisted.

[0094] like Figure 3This provides an example diagram of data segmentation loading and transmission. In this example, the second business data table has three fields: "FIL_NAM", "LON_NBR", and "LON_TAG". The actual business data includes related data such as "PLAXXDTAP" and "PLBXXDTAP", but after reconstruction, the business system only needs the "PLBXXDTAP" related data. Therefore, the required data is filtered during the pre-segmentation process.

[0095] Then, the data is segmented according to the unique index column LON_NBR, and the first record of the index column is taken. After the segmentation is completed, the complete data is queried and transmitted concurrently.

[0096] After the transmission is complete, the field names are converted according to the differences in table structure, and the actual meaning of "LON_TAG" is split into fields to obtain "small_tag" and "open_tag". This efficiently retrieves the complete data from the "original aggregated business table" and persists it.

[0097] Step S20: Determine the common table fields of the first business data table and the second business data table, and determine the comparison result between the first business data table and the second business data table based on the field values ​​of each common table field;

[0098] It should be noted that common table fields are fields in both tables that have the same name and data type. These common fields will be used for subsequent data comparison. For each set of common table fields, their field values ​​can be compared, and the comparison results can be recorded to determine whether the field values ​​are consistent.

[0099] Optionally, comparison results can be recorded using tables, reports, or other formats. Based on the comparison results, a report or dataset can be generated to record the consistency of all common table fields. Inconsistent field values ​​can be marked or highlighted, and the number of consistent and inconsistent fields can be counted for subsequent analysis.

[0100] After the "segmented loading and transmission" is completed, the "difference marking" processor is activated to perform autonomous comparison and marking of all fields in the table, flattening them into a "table field-level comparison file," such as... Figure 4 The table field-level comparison file structure is shown below. Flattening the structure facilitates data browsing, sorting, grouping, and statistical analysis, thus improving efficiency.

[0101] Understandably, by identifying common table fields and comparing field values, the consistency between the first business data table and the second business data table can be effectively verified.

[0102] Step S30: Determine whether there are any differing fields in the common table fields based on the comparison results;

[0103] It should be noted that the comparison results obtained from step S20 include two parts: a list of common table fields and a comparison of the field values ​​for each common table field. For each common table field, the field values ​​are checked to ensure they are completely consistent. If inconsistencies are found, they need to be marked or recorded.

[0104] Once the data table has been marked with differences, it can be immediately handed over to the "Difference Backtracker" for difference analysis. Figure 5 The table structure is configured to provide field difference hints. If the difference tracer finds field differences, it will immediately follow the instructions. Figure 5 The system retrieves possible causes of discrepancies between table fields, immediately notifies developers to track, investigate, and fix the issue, and updates the "Table Field Scoring Table" according to relevant rules if the fields match.

[0105] Understandably, after completing the data comparison, by examining the differences in the common table fields, issues with data consistency can be identified, which helps ensure data quality and provides a reliable basis for further data processing and decision-making.

[0106] Step S40: If there are no discrepancy fields, then the first business system is determined to have passed data verification.

[0107] It should be noted that in step S30, the common table fields between the first and second business data tables have been compared, and it has been determined whether there are any discrepancies. If the consistency of all common table fields is confirmed, i.e., no discrepancies exist, the first business system can be considered to have passed data verification. If discrepancies are found in the common table fields, the first business system fails data verification. In this case, it is necessary to further analyze the reasons for the discrepancies and take corresponding measures to resolve the problem.

[0108] Optionally, if no discrepancies are found in the fields, the table field scoring table needs to be updated according to the relevant rules. Figure 6 A table with scoring data is provided. Figure 7 A flowchart for updating the table field rating table is provided. ALL_RATE represents the number of times data matches during the entire comparison period / the total number of executions. When fields match, both field values ​​are incremented by 1; otherwise, the total number of executions is incremented by 1. RECENT_COUNT represents the total number of recent consecutive matches. When a difference exists, the count is initialized to 0; otherwise, it is incremented by 1.

[0109] Optionally, if the verification passes, the first business system (the restructured business system) can be officially switched to the production environment according to the plan. If the verification fails, further investigation and repair of the discrepancies are required until all issues are resolved.

[0110] like Figure 8 As shown, Figure 8 A diagram showing the relationship between system reconstruction, verification, and deployment is provided. Before reconstruction, the application service corresponds to the database before reconstruction, and after reconstruction, the application service corresponds to the database after reconstruction. A self-comparative analysis of the two databases is performed. Once the analysis results determine that the application service after reconstruction meets the switching criteria, the system can be switched over, gradually replacing the application service before reconstruction with the application service after reconstruction. This ensures that the application service after reconstruction can smoothly replace the original service, and that data consistency is guaranteed.

[0111] Understandably, determining the presence of discrepancies in fields is used to decide whether the first business system has passed data validation. If validation passes, it indicates that the restructured business system maintains data consistency with the pre-restructuring system, allowing for a safe system switchover. If validation fails, further analysis of the reasons for the discrepancies is needed, along with measures to resolve the issues to ensure data consistency and system stability.

[0112] This embodiment provides a data verification method based on a reconstructed system. It involves obtaining a first business data table from a first business system and a second business data table from a second business system that matches the first business data table. The first business system is the reconstructed business system, and the second business system is the pre-reconstruction business system. Common table fields between the first and second business data tables are determined, and a comparison result is calculated based on the values ​​of these common table fields. The comparison result is then used to determine if any discrepancies exist among the common table fields. If no discrepancies exist, the first business system passes data verification. This embodiment uses automated methods for data comparison, ensuring data consistency between the reconstructed and pre-reconstruction business systems. Because automation can quickly process large amounts of data, it reduces the verification time. Compared to traditional manual verification methods, it avoids potential human errors and improves the accuracy of data verification.

[0113] For example, to help understand the implementation flow of the data verification method based on the reconstructed system obtained by combining this embodiment with the above embodiment one, please refer to... Figure 9 , Figure 9 A simplified flowchart of a data verification method based on a reconstructed system is provided, specifically:

[0114] The application services and databases have been re-architected, but the business logic remains largely the same. The "data self-comparison and analysis" function is integrated into the restructured application services.

[0115] There are pre-reconstruction application services and post-reconstruction application services. The pre-reconstruction application service contains a pre-reconstruction database, which includes original business tables 1 through m. All business tables requiring verification are aggregated into an original aggregated business table. Similarly, the post-reconstruction application service contains a post-reconstruction database, which includes post-reconstruction business tables 1 through m. All business tables requiring verification are aggregated into a post-reconstruction aggregated business table.

[0116] First, the application service runs normally before reconstruction, while the reconstructed application service runs in parallel. Key nodes (such as accounting) simulate identical business data through a stencil. After both systems have completed execution, the "segmented loading and transmission" function filters, batch-transmits, and transforms the data from the "aggregated business table" before reconstruction, sending the original aggregated business table to the reconstructed application service via segmented loading and transmission. Then, the "difference marking" function is activated to scan and compare the data, flattening all data table fields and registering them in the "table field comparison data table" for subsequent grouping statistics, querying, and other operations.

[0117] Next, the "Difference Backtracker" will be activated. If a data discrepancy is found, the system will backtrack from the "Table Field Difference Hint Configuration" according to the table fields to check the possible causes of the discrepancy, notify relevant personnel, provide accurate hints, and intervene in a timely manner. If there is no discrepancy, the "Table Field Scoring Data Table" will be updated according to the rules.

[0118] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter. Based on this, after step S13, the data verification method based on the reconstructed system further includes steps S14 to S15:

[0119] Step S14: Identify the differences in table structure between the first business data table and the second business data table, wherein the differences in table structure include differences in field names and differences in field types;

[0120] Understandably, by identifying and recording differences in field names and field types, data consistency can be ensured during subsequent data validation, thereby guaranteeing the accuracy of subsequent data processing.

[0121] Step S15: Based on the table structure of the first business data table, convert the field names and field types of the second business data table that have different table structures, and update the second business data table based on the converted field names and field types.

[0122] It should be noted that, based on the field name differences identified in step S14, a field mapping table is created to map the field names in the second business data table to the field names in the first business data table. Using this field mapping table, the field names in the second business data table are updated to be consistent with those in the first business data table. Based on the field type differences identified in step S14, field type conversion rules are determined. According to these rules, inconsistent field types in the second business data table are converted. SQL statements are then used to modify the table structure of the second business data table to ensure consistency with the first business data table.

[0123] In one feasible implementation, after step S13, the data verification method based on the reconstructed system may further include steps A10 to A50:

[0124] Step A10: Check whether there are long fields in each field of the second business data table, where a long field is a field containing composite information or multiple attributes;

[0125] It should be noted that the detection of long fields in the second business data table is necessary. These long fields usually contain complex information or multiple attributes. Identifying these long fields is important for subsequent processing, such as breaking them down into more specific shorter fields.

[0126] Optionally, load the second business data table, examine its field structure, and determine the data type and length of each field. Determine the criteria for long fields—that is, which fields are considered long—by analyzing their content or length. For example, check if a field contains a specific delimiter (such as a comma or semicolon), or if its length exceeds a certain threshold. SQL statements can be used to find fields containing specific delimiters, or a programming language can be used to analyze field lengths.

[0127] Step A20: If a long field exists, determine the field content of the long field and determine the splitting rules of the long field by parsing the field content;

[0128] Optionally, the long field detected in step A10 is loaded, and its content is analyzed to determine the information it contains. For example, if the long field contains "YN", it is necessary to understand what "Y" and "N" represent respectively. Splitting criteria are defined based on the characteristics of the field content. For example, splitting rules can be defined based on specific delimiters (such as commas, semicolons, etc.) or fixed positions in the field content.

[0129] Step A30: Split the long field into multiple short fields according to the splitting rules, and name each short field to obtain the field name of each short field;

[0130] Optionally, the long field can be split according to the splitting rules defined in step A20. For example, if the content of the long field LON_TAG is "YN", it can be split into two short fields named small_tag and open_tag respectively.

[0131] Step A40: Create multiple new fields corresponding to the short fields in the second business data table, and fill the corresponding new fields with the field names and field contents of each short field;

[0132] Alternatively, a new field can be created by adding a column to the data table, and data manipulation methods can be used to populate the newly created field with the contents of the short field.

[0133] Step A50: Update the second business data table based on the field name and field content of the new field.

[0134] This implementation can break down long fields in the second business data table into more specific short fields, thereby improving the readability and processability of the data.

[0135] This embodiment identifies the table structure differences between the first and second business data tables, including differences in field names and field types. Based on the table structure of the first business data table, the corresponding field names and field types in the second business data table with the differences are converted, and the second business data table is updated based on the converted field names and types. This embodiment ensures the consistency of the table structure between the first and second business data tables. By identifying and processing table structure differences, data consistency during data verification can be ensured, thereby guaranteeing the accuracy of subsequent data processing.

[0136] Based on the first embodiment of this application, in the third embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter. Furthermore, step S40 may also include steps B10 to B50:

[0137] Step B10: Determine the number of fields in the common table fields of the first business data table and the second business data table;

[0138] Step B20: Take the field value corresponding to each common table field in the first business data table as the first field value, and take the field value corresponding to each common table field in the second business data table as the second field value;

[0139] Step B30: Obtain the number of times each first field value and its corresponding second field value match within a preset comparison time.

[0140] Step B40: Determine whether the system switching conditions are met based on the number of fields in the common table and the number of times the value of each first field matches the value of the corresponding second field within a preset comparison time.

[0141] It should be noted that the self-comparison of data has a time requirement, and a suitable preset comparison time needs to be defined. Define the threshold for the number of consecutive comparisons of RECENT_COUNT as t. Once the threshold is reached, the field can be considered to have been verified as correct.

[0142] The system handover maturity can be calculated based on the number of fields in the common table and the number of times each first field value matches its corresponding second field value within a preset comparison time. The formula for calculating system handover maturity is as follows:

[0143]

[0144] Where W represents the system switching maturity, n represents the number of fields in the common table, and RECENT_COUNT represents the number of times each first field value matches the corresponding second field value within a preset comparison time.

[0145] Figure 10 A system switchover maturity curve is provided. It can be seen that when the system switchover maturity reaches 1, the system can be considered to have met the switchover criteria, representing the best-case scenario. In the normal and worst-case scenarios, the system switchover maturity gradually increases from the lowest level. The longer W remains at 0, the more frequently abnormal situations are compared within the current time period.

[0146] Step B50: If the number of fields in the common table and the number of times each first field value matches the corresponding second field value within the preset comparison time are the same, then the system switching conditions are met, and the first business system is confirmed to have passed data verification.

[0147] It should be noted that if the number of fields in the common table and the number of times the first field value matches the corresponding second field value within the preset comparison time are the same, it is equivalent to the above system switching maturity reaching 1.

[0148] In one feasible implementation, steps B60 to B70 may be included after step B50:

[0149] Step B60: Gradually switch business traffic destined for the second business system to the first business system until all business traffic is directed to the first business system;

[0150] Optionally, the load balancer can be configured to gradually route more requests to the primary business system, increasing the traffic to the primary business system in a predetermined manner.

[0151] Step B70: Gradually shut down the system services of the second business system.

[0152] Use system management tools or command-line tools to gradually shut down the various services of the second business system.

[0153] This implementation method can gradually switch business traffic to the first business system and gradually shut down the system services of the second business system after the system switching conditions are met, thus ensuring business continuity and a smooth transition of system switching.

[0154] This embodiment determines the number of common fields in the first and second business data tables; uses the field values ​​corresponding to each common field in the first business data table as first field values, and the field values ​​corresponding to each common field in the second business data table as second field values; and obtains the number of times each first field value and its corresponding second field value match within a preset comparison time. Based on the number of common fields and the number of times each first field value matches its corresponding second field value within the preset comparison time, it determines whether the system switching conditions are met. If the number of common fields and the number of times each first field value matches its corresponding second field value within the preset comparison time are the same, then the system switching conditions are met, and the first business system passes data verification. This embodiment provides a method for determining system switching conditions, which not only improves the accuracy and reliability of data verification but also simplifies the determination of system switching conditions, ensures the security and stability of system switching, and enhances the automation level of system switching.

[0155] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the data verification method based on the reconstructed system in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0156] This application also provides a data verification device based on a reconstruction system; please refer to [reference needed]. Figure 11 The data verification device based on the reconstruction system includes:

[0157] The acquisition module 10 is used to acquire a first business data table in a first business system and acquire a second business data table that matches the first business data table from a second business system, wherein the first business system is the reconstructed business system and the second business system is the business system before reconstruction.

[0158] The comparison module 20 is used to determine the common table fields of the first business data table and the second business data table, and to determine the comparison result of the first business data table and the second business data table based on the field values ​​of each of the common table fields.

[0159] The verification module 30 is used to determine whether there are any differing fields in the common table fields based on the comparison results.

[0160] The verification module 40 is used to determine that the first business system passes data verification if the difference field does not exist.

[0161] The data verification device based on a reconstruction system provided in this application, employing the data verification method based on a reconstruction system in the above embodiments, can solve the technical problem of how to improve the accuracy of data verification in a reconstruction system. Compared with the prior art, the beneficial effects of the data verification device based on a reconstruction system provided in this application are the same as those of the data verification method based on a reconstruction system provided in the above embodiments, and other technical features in the data verification device based on a reconstruction system are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0162] This application provides a data verification device based on a reconstruction system. The data verification device based on a reconstruction system includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the data verification method based on the reconstruction system in the first embodiment described above.

[0163] The following is for reference. Figure 12 This document illustrates a structural diagram of a data verification device based on a reconstruction system suitable for implementing embodiments of this application. The data verification device based on a reconstruction system in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 12 The data verification device based on the reconstruction system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0164] like Figure 12As shown, the data verification device based on the reconstruction system may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the data verification device based on the reconstruction system. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the data verification device based on the reconfiguration system to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a data verification device based on a reconfiguration system with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented or possessed alternatively.

[0165] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0166] The data verification device based on the reconstruction system provided in this application, employing the data verification method based on the reconstruction system in the above embodiments, can solve the technical problem of how to improve the accuracy of data verification in the reconstruction system. Compared with the prior art, the beneficial effects of the data verification device based on the reconstruction system provided in this application are the same as the beneficial effects of the data verification method based on the reconstruction system provided in the above embodiments, and other technical features in the data verification device based on the reconstruction system are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0167] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0168] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0169] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the data verification method based on the reconstruction system in the above embodiments.

[0170] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the data verification method based on the reconstruction system described above, thereby solving the technical problem of how to improve the accuracy of data verification in the reconstruction system. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the data verification method based on the reconstruction system provided in the above embodiments, and will not be repeated here.

[0171] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data verification method based on the reconstructed system described above.

[0172] The computer program product provided in this application can solve the technical problem of how to improve the accuracy of data verification in a reconstruction system. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the data verification method based on the reconstruction system provided in the above embodiments, and will not be repeated here.

[0173] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A data verification method based on a reconstructed system, characterized in that, The data verification method based on the reconstructed system, applied to the first business system, includes: Obtain a first business data table from a first business system, and obtain a second business data table from a second business system that matches the first business data table, wherein the first business system is the restructured business system, and the second business system is the business system before restructuring; the step of obtaining the second business data table from the second business system that matches the first business data table includes: A data segmentation request is sent to the second business system. The data segmentation request includes the first business data table and data segmentation rules. The second business system receives the data segmentation request, determines a second business data table that matches the first business data table based on the first business data table in the data segmentation request, segments the second business data table based on the data segmentation rules, determines multiple segmented data, and generates an index list based on the unique index of the first data entry in each segmented data. The index list is then sent to the first business system. The system receives an index list sent by the second business system, generates multiple data acquisition requests based on multiple unique indexes in the index list, and sends each data acquisition request to the second business system. The second business system receives each data acquisition request, acquires segmented data corresponding to each unique index based on each data acquisition request and the unique index corresponding to the data acquisition request, and sends each segmented data to the first business system. Receive multiple segments of data sent by the second business system, and merge the segments of data to obtain a second business data table; Determine the common table fields of the first business data table and the second business data table, and determine the comparison result between the first business data table and the second business data table based on the field values ​​of each common table field; Based on the comparison results, determine whether there are any differing fields in the common table fields; If the difference field does not exist, then the first business system is determined to have passed data verification.

2. The data verification method based on the reconstructed system as described in claim 1, characterized in that, The step of sending a data segmentation request to the second business system includes: The number of data entries for each segment of data is determined based on the first business data table, and the number of data entries for each segment of data is used as the data segmentation rule. A data segmentation request is generated based on the data segmentation rules and the first business data. The data segmentation request is sent to the second business system. The second business system receives the data segmentation request, matches a target data table in the second business system according to the first business data table in the data segmentation request, filters the corresponding data entries in the first business data table in the target data table, determines the second business data table based on all the filtered data entries, divides the second business data table into multiple segments according to the number of data entries in the data segmentation rules, generates an index list based on the unique index of the first data entry in each segment, and sends the index list to the first business system.

3. The data verification method based on the reconstructed system as described in claim 1, characterized in that, After the step of merging the segmented data to obtain the second business data table, the method further includes: Identify the table structure differences between the first business data table and the second business data table, wherein the table structure differences include differences in field names and differences in field types; Based on the table structure of the first business data table, the field names and field types of the second business data table with different table structures are converted, and the second business data table is updated based on the converted field names and field types.

4. The data verification method based on the reconstructed system as described in claim 3, characterized in that, After the step of merging the segmented data to obtain the second business data table, the method further includes: Detect whether there are long fields in each field of the second business data table, wherein the long field is a field containing composite information or multiple attributes; If the long field exists, determine the field content of the long field, and determine the splitting rule of the long field by parsing the field content; The long field is split into multiple short fields according to the splitting rules, and each short field is named to obtain the field name of each short field; In the second business data table, create multiple new fields corresponding to the short field, and fill the corresponding new fields with the field name and field content of each short field; Update the second business data table according to the field name and field content of the new field.

5. The data verification method based on the reconstructed system as described in claim 1, characterized in that, The step of determining that the first business system has passed data verification further includes: Determine the number of fields in the common table fields of the first business data table and the second business data table; The field values ​​corresponding to the common table fields in the first business data table are used as the first field values, and the field values ​​corresponding to the common table fields in the second business data table are used as the second field values. Within a preset comparison time, obtain the number of times each of the first field values ​​and the corresponding second field values ​​are consistent. Whether the system switching conditions are met is determined based on the number of fields in the common table and the number of times each first field value matches the corresponding second field value within the preset comparison time. If the number of fields in the common table and the number of times each first field value matches the corresponding second field value within the preset comparison time are the same, then the system switching conditions are met, and the first business system is determined to have passed data verification.

6. The data verification method based on the reconstructed system as described in claim 5, characterized in that, After determining that the first business system has passed data verification, the method further includes: The business traffic destined for the second business system will be gradually switched to the first business system until all business traffic is directed to the first business system; Gradually shut down the system services of the second business system.

7. A data verification device based on a reconstruction system employing the method described in claim 1, characterized in that, The data verification device based on the reconstruction system includes: The acquisition module is configured to acquire a first business data table from a first business system and acquire a second business data table from a second business system that matches the first business data table, wherein the first business system is the reconstructed business system and the second business system is the business system before reconstruction; the step of acquiring the second business data table from the second business system that matches the first business data table includes: A data segmentation request is sent to the second business system. The data segmentation request includes the first business data table and data segmentation rules. The second business system receives the data segmentation request, determines a second business data table that matches the first business data table based on the first business data table in the data segmentation request, segments the second business data table based on the data segmentation rules, determines multiple segmented data, and generates an index list based on the unique index of the first data entry in each segmented data. The index list is then sent to the first business system. The system receives an index list sent by the second business system, generates multiple data acquisition requests based on multiple unique indexes in the index list, and sends each data acquisition request to the second business system. The second business system receives each data acquisition request, acquires segmented data corresponding to each unique index based on each data acquisition request and the unique index corresponding to the data acquisition request, and sends each segmented data to the first business system. Receive multiple segments of data sent by the second business system, and merge the segments of data to obtain a second business data table; The comparison module is used to determine the common table fields of the first business data table and the second business data table, and to determine the comparison result of the first business data table and the second business data table based on the field values ​​of each common table field. The verification module is used to determine whether there are any differing fields in the common table fields based on the comparison results. The verification module is used to determine that the first business system passes data verification if the difference field does not exist.

8. A data verification device based on a reconstruction system, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the data verification method based on the reconstructed system as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the data verification method based on the reconstructed system as described in any one of claims 1 to 6.

Images