A data checking system and method applied in data exchange

Abstract

The application discloses a data checking system and method applied to data exchange, and the method comprises the following steps: dividing exchange data into batches in a starting data exchange node, generating batch data and batch information; updating batch information of a current data exchange node; transmitting the exchange data and the updated batch information to a next data exchange node; repeating until the exchange data is transmitted to a final data exchange node; checking all the updated batch information, and analyzing to obtain problems in a data exchange transmission process. The application can automatically and quickly know data transmission conditions, find data loss problems, locate fault nodes, and know approximate causes of faults.

Description

Technical Field

[0001] This invention belongs to the field of data exchange technology, specifically relating to a data verification system and method applied in data exchange. Background Technology

[0002] With the continuous development of social informatization, the business systems and data of various industries are becoming increasingly massive. Data exchange between multiple systems is increasing, and the exchange paths are becoming longer. Especially in the field of e-government, due to the large number of business systems, the large volume of exchanged data, inconsistent data standards, and complex network conditions, various regions have even planned and built dedicated data exchange platforms to facilitate data exchange between departments.

[0003] In actual data transmission across multiple data exchange nodes, data transmission may be interrupted or partially lost due to hardware failures of exchange node servers and networks, as well as software issues such as operating systems, databases, and exchange platforms. Incomplete or inaccurate data can cause difficulties for applications and become a pain point for business applications. To avoid these problems, currently, in complex cross-node data exchange processes, it is often necessary to have dedicated maintenance personnel to manually monitor and verify the data transmission status, promptly identify data problems, and locate the problematic nodes. However, manual maintenance is inevitably costly and inefficient. Summary of the Invention

[0004] To address the shortcomings of the existing technologies, the present invention aims to provide a data verification system and method for data exchange, thereby solving the problem that currently, in complex cross-node data exchange processes, dedicated maintenance personnel are required to manually monitor and verify data transmission. The present invention can automatically and quickly obtain information on data transmission, detect data loss problems, locate faulty nodes, and determine the approximate cause of the fault.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] The present invention provides a data verification system for data exchange, comprising:

[0007] The batch partitioning module is used to partition the exchanged data into batches, generating batch data and batch information;

[0008] The update module is used to update batch information and transmit the updated batch information to the data verification module;

[0009] The transmission module is used to transmit batch data and batch information to each data exchange node;

[0010] The data verification module is used to verify batch information and identify problems during the data exchange and transmission process.

[0011] The present invention provides a data verification method for data exchange, based on the above system, comprising the following steps:

[0012] 1) Divide the exchanged data into batches in the initial data exchange node, and generate batch data and batch information;

[0013] 2) Update the batch information of the current data exchange node;

[0014] 3) Transmit the exchanged data and updated batch information to the next data exchange node;

[0015] 4) Repeat steps 2) and 3) until the exchanged data is transmitted to the final data exchange node;

[0016] 5) Verify all updated batch information and analyze the problems in the data exchange and transmission process.

[0017] Further, step 1) specifically includes:

[0018] 11) Set the exchange data Data to have been transmitted from the data exchange initiating system to the starting data exchange node Node_A, that is, Node_A already has the exchange data Data;

[0019] 12) The batch partitioning module in the starting data exchange node Node_A divides the exchanged data into batches according to the network, database, and exchanged data, based on the amount of data that can be correctly transmitted at one time and the file size, to obtain batch data and batch information. The batch information includes batch code (Batch_Code), batch feature code (Batch_Hash), batch data size (Batch_Size), and batch code table (Batch_Table).

[0020] 13) The batch segmentation module extracts each batch of data one by one, and uses each batch of data and its corresponding batch information to form an independent exchange data packet. The batch information includes the batch code, batch feature code, batch data volume and batch code table.

[0021] Furthermore, step 12) specifically includes:

[0022] 121) The exchanged data is split into several small data packets, each data packet being a batch of data Data_1, Data_2, Data_3, ..., Data_N, and stored separately;

[0023] 122) Generate corresponding batch codes Batch_1_Code, Batch_2_Code, Batch_3_Code, ..., Batch_N_Code for each batch of data Data_1, Data_2, Data_3, ..., Data_N; the batch codes are generated using a unique code generation algorithm (such as UUID, NanoID, etc.) based on the number of batches; different batches do not repeat the same codes.

[0024] 123) For each batch of data Data_1, Data_2, Data_3, ..., Data_N, generate corresponding batch feature codes Batch_1_Hash, Batch_2_Hash, Batch_3_Hash, ..., Batch_N_Hash; the batch feature codes are generated using hash algorithms (such as national cryptographic standards SM3, MD5, CRC, SHA, etc.) based on each batch of data; different data exchange nodes will calculate the same batch feature codes when using the same batch of data and hash algorithms;

[0025] 124) For each batch of data Data_1, Data_2, Data_3, ..., Data_N, generate the corresponding batch data size Batch_1_Size, Batch_2_Size, Batch_3_Size, ..., Batch_N_Size; the batch data size represents the data size, which is the number of data records or the file size;

[0026] 125) Arrange all batch codes in order according to the batch sequence (Batch_1_Code, Batch_2_Code, Batch_3_Code, ..., Batch_N_Code) to form a batch code table (Batch_Table) containing all batch codes.

[0027] Furthermore, the batch division in step 12) is based on different original data types, as follows:

[0028] 126) If the original data is a data table in a relational database, the batches should be divided according to the number of records in the data table; use a fixed number of records (e.g., 5000 records) to divide the batches, and the number of records in each batch after division should be less than or equal to the fixed number of records; the fixed number of records should be modified according to the actual network architecture and transmission conditions;

[0029] 127) If the original data is a large file, the batches should be divided according to the file size; use a fixed storage size (e.g., 100M) to divide the batches, and the size of each batch of data after division should be less than or equal to the fixed storage size;

[0030] 128) If the original data consists of multiple small files, batches should be divided into individual files; after division, each file constitutes a batch.

[0031] Further, the exchange data packets in step 13) are: (Data_1, Batch_1_Code, Batch_1_Hash, Batch_1_Size, Batch_Table), (Data_2, Batch_2_Code, Batch_2_Hash, Batch_2_Size, Batch_Table), (Data_3, Batch_3_Code, Batch_3_Hash, Batch_3_Size, Batch_Table), ..., (Data_N, Batch_N_Code, Batch_N_Hash, Batch_N_Size, Batch_Table); thus splitting the original exchange data Data into N independent small exchange data packets.

[0032] Furthermore, step 2) specifically includes:

[0033] 21) The current data exchange node receives an exchange data packet (Data_n, Batch_n_Code, Batch_n_Hash, Batch_n_Size, Batch_Table) containing batch data and batch information, where n is 1, 2, 3, ..., N, that is, any exchange data packet after batching. The exchange data packet can come from this data exchange node or other data exchange nodes.

[0034] 22) The update module determines whether the received batch data and batch information exchange data packets come from the current data exchange node; if they come from the current data exchange node, proceed to step 26); if they come from other data exchange nodes, proceed to step 23.

[0035] 23) The update module regenerates the corresponding batch signature code Batch_n_Hash_New for the received batch data Data_n (to distinguish whether the received signature is old or newly generated, the suffix New is added to the newly generated Batch_n_Hash); the new batch signature code is generated based on each batch of data using a hash algorithm (such as Chinese cryptographic SM3, MD5, CRC, SHA, etc.); different data exchange nodes will calculate the same batch signature code when using the same batch data and algorithm;

[0036] 24) The update module regenerates the corresponding batch data size Batch_n_Size_New for the received batch data Data_n (to distinguish whether the received data is old or newly generated, the suffix New is added to the newly generated Batch_n_Size); the new batch data size represents the data size, which is the number of data records or the file size;

[0037] 25) The update module updates the batch feature code Batch_n_Hash and batch data size Batch_n_Size in the batch information to form new batch information (Batch_n_Code, Batch_n_Hash_New, Batch_n_Size_New, Batch_Table); (In the following text, Batch_n_Hash_New and Batch_n_Size_New will no longer distinguish whether they have been updated, and will be uniformly marked as Batch_n_Hash and Batch_n_Size).

[0038] 26) The update module combines the node number and the updated batch information into an exchange data packet (Node_n, Batch_n_Code, Batch_n_Hash, Batch_n_Size, Batch_Table) and sends it to the data verification module.

[0039] Furthermore, step 3) specifically includes:

[0040] 31) The transmission module of the current data exchange node combines the exchange data, node number, and batch information into an exchange data packet (Data_n, Node_n, Batch_n_Code, Batch_n_Hash, Batch_n_Size, Batch_Table) and sends it to the next data exchange node according to the data transmission needs.

[0041] Furthermore, in step 31), the data transmission module uses different implementation methods depending on the different batch data types:

[0042] 311) If the batch data type is a relational database, the data transfer module uses a data extraction ETL tool, a database job, or an extraction program.

[0043] 312) If the batch data type is a file, the data transfer module uses a message queue, FTP tool, operating system transfer job, or control transfer program.

[0044] Furthermore, step 4) specifically includes:

[0045] 41) Repeat steps 2) and 3) until the exchange data packet, which combines the exchange data and batch information, is delivered to the final data exchange node (Node_F);

[0046] 42) If all batches of exchanged data packets are transmitted correctly, then the final data exchange node (Node_F) receives an exchanged data packet consisting of all the exchanged data from batches 1, 2, 3, ..., N, and the updated batch information.

[0047] 43) The update module of the final data exchange node (Node_F) uses the batch number in the updated batch information to determine whether the data transmission is correct; if the data transmission is correct, proceed to step 44); if the data transmission is incorrect, stop running and issue a notification.

[0048] 44) The batch partitioning module of the final data exchange node (Node_F) assembles each batch of data according to the batch number sequence and batch data in the batch code table Batch_Table in any batch information, and restores the original data before the batch splitting in step 1), so as to complete the transfer of the original data from the initial data exchange node Node_A to the final data exchange node Node_F.

[0049] Further, step 44) specifically includes:

[0050] 441) If the batch data type is a relational database, the order of the batches will not be considered during restoration, and the data will be directly inserted into the target data table;

[0051] 442) If the batch data type is a large file, during restoration, the batch data is assembled sequentially according to the batch number sequence in the Batch_Table, and finally restored to the original data;

[0052] 443) If the batch data type consists of multiple small files, the order of the batches will not be considered during restoration. The batch data will be directly restored to the original files and placed in the target folder.

[0053] Furthermore, step 5) specifically includes:

[0054] 51) The data verification module receives and stores batch information sent by each data exchange node in real time;

[0055] 52) The data verification module uses a single batch number as a clue to cross-compare batch information with the same batch number sent by different data exchange nodes in order to discover that single batch exchange data is lost or tampered with during transmission between data exchange nodes.

[0056] 53) The data verification module uses the batch code table Batch_Table in the batch information as a clue to cross-compare the batch information split from the same original data sent by a data exchange node to determine whether the original data transmission is correct.

[0057] Further, step 52) specifically includes:

[0058] 521) Determine whether the batch data sizes Batch_1_Size, Batch_2_Size, Batch_3_Size, ..., Batch_n_Size in the batch information sent by different data exchange nodes with the same batch number are all consistent. If they are consistent, it means that the corresponding batch data was transmitted correctly in each data exchange node; if they are inconsistent, it means that there is a data loss error. Further analyze the batch data size of the exchange node that has changed to obtain the specific exchange node that has made a mistake.

[0059] 522) Determine whether the batch feature codes Batch_1_Hash, Batch_2_Hash, Batch_3_Hash, ..., Batch_n_Hash in the batch information sent by different data exchange nodes with the same batch number are all consistent. If they are consistent, it means that the corresponding batch data was transmitted correctly in each data exchange node; if they are inconsistent, it means that some data has been tampered with. Further analyze the batch feature codes of the exchange nodes that have changed to obtain the specific exchange node that malfunctioned.

[0060] Furthermore, step 53) specifically includes:

[0061] 531) Based on the batch number in the Batch_Table, determine whether the data exchange node involved in the data transmission is missing the corresponding batch information; if so, it means that the original data lost several complete batches of data during transmission.

[0062] 532) Based on the batch number existing in the Batch_Table, query the verification result in step 52); if there is an error, it means that the original data was lost or tampered with during transmission.

[0063] The beneficial effects of this invention are:

[0064] The method of the present invention can reduce the failure rate of data loss during data exchange, improve data exchange efficiency, quickly know the data transmission status, discover data loss problems, locate faulty nodes, and know the approximate cause of the fault in scenarios where data is transmitted across multiple data exchange nodes. Attached Figure Description

[0065] Figure 1 This is a schematic diagram of the method of the present invention. Detailed Implementation

[0066] The following are explanations of some terms used in this invention:

[0067] Data exchange node: refers to a software system and server that can completely receive data from other data exchange nodes, completely store data, and transmit data to the next data exchange node. It is represented by the Node symbol. Specifically, Node_A represents data exchange node A, Node_B represents data exchange node B, and so on.

[0068] Data exchange: refers to symbolic information that can be processed by computers and needs to be completely transmitted between different data exchange nodes, represented by the word "Data".

[0069] Data transfer: refers to the process of exchanging data by completely copying it from one data exchange node to another using software with data replication capabilities.

[0070] Data exchange: refers to the process of exchanging data and transmitting it multiple times between multiple data exchange nodes.

[0071] Batch: A label for the exchanged data in a single data transmission, to facilitate the representation and verification of the exchanged data. Batch is used to represent batch information, and Batch_01, Batch_02, Batch_03, ..., Batch_n represent the batch information for each N batches.

[0072] Batch partitioning: If the amount of exchanged data is extremely large, various problems can easily arise during a single data transfer, leading to frequent failures. Therefore, large amounts of exchanged data need to be divided into smaller amounts for transmission to reduce the failure rate and facilitate tracking and verification. This process is called batch partitioning. For example, if the amount of exchanged data (Data) is 1 million records, it is too large to be transmitted in a single transfer. Therefore, it is partitioned into 10 batches, each containing 100,000 records. This article uses Data_01, Data_02, Data_03, ..., Data_09, Data_10 to represent these 10 batches, where Data_n represents the exchanged data corresponding to any one of the batches.

[0073] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to examples and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0074] The present invention provides a data verification system for data exchange, comprising:

[0075] The batch partitioning module is used to partition the exchanged data into batches, generating batch data and batch information;

[0076] The update module is used to update batch information and transmit the updated batch information to the data verification module;

[0077] The transmission module is used to transmit batch data and batch information to each data exchange node;

[0078] The data verification module is used to verify batch information and identify problems during the data exchange and transmission process.

[0079] Reference Figure 1 As shown, a data verification method for data exchange according to the present invention, based on the above system, comprises the following steps:

[0080] 1) In the initial data exchange node, the exchanged data is divided into batches, generating batch data and batch information; specifically including:

[0081] 11) Set the exchange data Data to have been transmitted from the data exchange initiating system to the starting data exchange node Node_A, that is, Node_A already has the exchange data Data;

[0082] 12) The batch partitioning module in the starting data exchange node Node_A divides the exchanged data into batches according to the network, database, and exchanged data, based on the amount of data that can be correctly transmitted at one time and the file size, to obtain batch data and batch information. The batch information includes batch code (Batch_Code), batch feature code (Batch_Hash), batch data size (Batch_Size), and batch code table (Batch_Table).

[0083] 13) The batch segmentation module extracts each batch of data one by one, and uses each batch of data and its corresponding batch information to form an independent exchange data packet. The batch information includes the batch code, batch feature code, batch data volume and batch code table.

[0084] Specifically, step 12) includes:

[0085] 121) The exchanged data is split into several small data packets, each data packet being a batch of data Data_1, Data_2, Data_3, ..., Data_N, and stored separately;

[0086] 122) Generate corresponding batch codes Batch_1_Code, Batch_2_Code, Batch_3_Code, ..., Batch_N_Code for each batch of data Data_1, Data_2, Data_3, ..., Data_N; the batch codes are generated using a unique code generation algorithm (such as UUID, NanoID, etc.) based on the number of batches; different batches do not repeat the same codes.

[0087] 123) For each batch of data Data_1, Data_2, Data_3, ..., Data_N, generate corresponding batch feature codes Batch_1_Hash, Batch_2_Hash, Batch_3_Hash, ..., Batch_N_Hash; the batch feature codes are generated using hash algorithms (such as national cryptographic standards SM3, MD5, CRC, SHA, etc.) based on each batch of data; different data exchange nodes will calculate the same batch feature codes when using the same batch of data and hash algorithms;

[0088] 124) For each batch of data Data_1, Data_2, Data_3, ..., Data_N, generate the corresponding batch data size Batch_1_Size, Batch_2_Size, Batch_3_Size, ..., Batch_N_Size; the batch data size represents the data size, which is the number of data records or the file size;

[0089] 125) Arrange all batch codes in order according to the batch sequence (Batch_1_Code, Batch_2_Code, Batch_3_Code, ..., Batch_N_Code) to form a batch code table (Batch_Table) containing all batch codes.

[0090] In step 12), the batch division is based on different original data types, as follows:

[0091] 126) If the original data is a data table in a relational database, the batches should be divided according to the number of records in the data table; use a fixed number of records (e.g., 5000 records) to divide the batches, and the number of records in each batch after division should be less than or equal to the fixed number of records; the fixed number of records should be modified according to the actual network architecture and transmission conditions;

[0092] 127) If the original data is a large file, the batches should be divided according to the file size; use a fixed storage size (e.g., 100M) to divide the batches, and the size of each batch of data after division should be less than or equal to the fixed storage size;

[0093] 128) If the original data consists of multiple small files, batches should be divided into individual files; after division, each file constitutes a batch.

[0094] In step 13), the exchange data packets are: (Data_1, Batch_1_Code, Batch_1_Hash, Batch_1_Size, Batch_Table), (Data_2, Batch_2_Code, Batch_2_Hash, Batch_2_Size, Batch_Table), (Data_3, Batch_3_Code, Batch_3_Hash, Batch_3_Size, Batch_Table), ..., (Data_N, Batch_N_Code, Batch_N_Hash, Batch_N_Size, Batch_Table); thus splitting the original exchange data Data into N independent small exchange data packets.

[0095] 2) Update the batch information of the current data exchange node; specifically including:

[0096] 21) The current data exchange node receives an exchange data packet (Data_n, Batch_n_Code, Batch_n_Hash, Batch_n_Size, Batch_Table) containing batch data and batch information, where n is 1, 2, 3, ..., N, that is, any exchange data packet after batching. The exchange data packet can come from this data exchange node or other data exchange nodes.

[0097] 22) The update module determines whether the received batch data and batch information exchange data packets come from the current data exchange node; if they come from the current data exchange node, proceed to step 26); if they come from other data exchange nodes, proceed to step 23.

[0098] 23) The update module regenerates the corresponding batch signature code Batch_n_Hash_New for the received batch data Data_n (to distinguish whether the received signature is old or newly generated, the suffix New is added to the newly generated Batch_n_Hash); the new batch signature code is generated based on each batch of data using a hash algorithm (such as Chinese cryptographic SM3, MD5, CRC, SHA, etc.); different data exchange nodes will calculate the same batch signature code when using the same batch data and algorithm;

[0099] 24) The update module regenerates the corresponding batch data size Batch_n_Size_New for the received batch data Data_n (to distinguish whether the received data is old or newly generated, the suffix New is added to the newly generated Batch_n_Size); the new batch data size represents the data size, which is the number of data records or the file size;

[0100] 25) The update module updates the batch feature code Batch_n_Hash and batch data size Batch_n_Size in the batch information to form new batch information (Batch_n_Code, Batch_n_Hash_New, Batch_n_Size_New, Batch_Table); (In the following text, Batch_n_Hash_New and Batch_n_Size_New will no longer distinguish whether they have been updated, and will be uniformly marked as Batch_n_Hash and Batch_n_Size).

[0101] 26) The update module combines the node number and the updated batch information into an exchange data packet (Node_n, Batch_n_Code, Batch_n_Hash, Batch_n_Size, Batch_Table) and sends it to the data verification module.

[0102] 3) Transmit the exchanged data and updated batch information to the next data exchange node; specifically including:

[0103] 31) The transmission module of the current data exchange node combines the exchange data, node number, and batch information into an exchange data packet (Data_n, Node_n, Batch_n_Code, Batch_n_Hash, Batch_n_Size, Batch_Table) and sends it to the next data exchange node according to the data transmission needs.

[0104] The data transmission module uses different implementation methods depending on the different batch data types:

[0105] 311) If the batch data type is a relational database, the data transfer module uses a data extraction ETL tool, a database job, or an extraction program.

[0106] 312) If the batch data type is a file, the data transfer module uses a message queue, FTP tool, operating system transfer job, or control transfer program.

[0107] 4) Repeat steps 2) and 3) until the exchanged data is transmitted to the final data exchange node; specifically including:

[0108] 41) Repeat steps 2) and 3) until the exchange data packet, which combines the exchange data and batch information, is delivered to the final data exchange node (Node_F);

[0109] 42) If all batches of exchanged data packets are transmitted correctly, then the final data exchange node (Node_F) receives exchanged data packets consisting of all exchanged data from batches 1, 2, 3, ..., N, and the updated batch information.

[0110] 43) The update module of the final data exchange node (Node_F) uses the batch number in the updated batch information to determine whether the data transmission is correct; if the data transmission is correct, proceed to step 44); if the data transmission is incorrect, stop running and issue a notification.

[0111] 44) The batch partitioning module of the final data exchange node (Node_F) assembles each batch of data according to the batch number sequence and batch data in the batch code table Batch_Table in any batch information, and restores the original data before the batch splitting in step 1), so as to complete the transfer of the original data from the initial data exchange node Node_A to the final data exchange node Node_F.

[0112] Specifically, step 44) includes:

[0113] 441) If the batch data type is a relational database, the order of the batches will not be considered during restoration, and the data will be directly inserted into the target data table;

[0114] 442) If the batch data type is a large file, during restoration, the batch data is assembled sequentially according to the batch number sequence in the Batch_Table, and finally restored to the original data;

[0115] 443) If the batch data type consists of multiple small files, the order of the batches will not be considered during restoration. The batch data will be directly restored to the original files and placed in the target folder.

[0116] 5) Verify all updated batch information and analyze any problems encountered during the data exchange and transmission process; specifically including:

[0117] 51) The data verification module receives and stores batch information sent by each data exchange node in real time;

[0118] 52) The data verification module uses a single batch number as a clue to cross-compare batch information with the same batch number sent by different data exchange nodes in order to discover that single batch exchange data is lost or tampered with during transmission between data exchange nodes.

[0119] 53) The data verification module uses the batch code table Batch_Table in the batch information as a clue to cross-compare the batch information split from the same original data sent by a data exchange node to determine whether the original data transmission is correct.

[0120] Specifically, step 52) includes:

[0121] 521) Determine whether the batch data sizes Batch_1_Size, Batch_2_Size, Batch_3_Size, ..., Batch_n_Size in the batch information sent by different data exchange nodes with the same batch number are all consistent. If they are consistent, it means that the corresponding batch data was transmitted correctly in each data exchange node; if they are inconsistent, it means that there is a data loss error. Further analyze the batch data size of the exchange node that has changed to obtain the specific exchange node that has made a mistake.

[0122] 522) Determine whether the batch feature codes Batch_1_Hash, Batch_2_Hash, Batch_3_Hash, ..., Batch_n_Hash in the batch information sent by different data exchange nodes with the same batch number are all consistent. If they are consistent, it means that the corresponding batch data was transmitted correctly in each data exchange node; if they are inconsistent, it means that some data has been tampered with. Further analyze the batch feature codes of the exchange nodes that have changed to obtain the specific exchange node that malfunctioned.

[0123] Specifically, step 53) includes:

[0124] 531) Based on the batch number in the Batch_Table, determine whether the data exchange node involved in the data transmission is missing the corresponding batch information; if so, it means that the original data lost several complete batches of data during transmission.

[0125] 532) Based on the batch number existing in the Batch_Table, query the verification result in step 52); if there is an error, it means that the original data was lost or tampered with during transmission.

[0126] This invention has many specific applications. The above description is only a preferred embodiment of this invention. It should be noted that for those skilled in the art, several improvements can be made without departing from the principle of this invention, and these improvements should also be considered within the scope of protection of this invention.

Claims

1. A data verification system applied in data exchange, characterized in that, include: The batch partitioning module is used to partition the exchanged data into batches, generating batch data and batch information; The update module is used to update batch information and transmit the updated batch information to the data verification module; The transmission module is used to transmit batch data and batch information to each data exchange node; The data verification module is used to verify batch information and identify problems during the data exchange and transmission process. The updated batch information specifically includes: The current data exchange node receives an exchange data packet (Data_n, Batch_n_Code, Batch_n_Hash, Batch_n_Size, Batch_Table) containing batch data and batch information, where n is 1, 2, 3, ..., N, representing any batch of exchange data packets. The exchange data packet can originate from this data exchange node or other data exchange nodes; Data_n represents the nth batch of data; Batch_n_Code is the batch code corresponding to the nth batch of data; Batch_n_Hash is the batch feature code corresponding to the nth batch of data; Batch_n_Size is the batch size corresponding to the nth batch of data; and Batch_Table is the batch code table. The update module determines whether the received batch data and batch information exchange data packet comes from the current data exchange node. If it comes from the current data exchange node, the update module combines the node number and the updated batch information into an exchange data packet (Node_n, Batch_n_Code, Batch_n_Hash, Batch_n_Size, Batch_Table) and sends it to the data verification module. If it comes from other data exchange nodes, the update module regenerates the corresponding batch feature code Batch_n_Hash_New for the received batch data Data_n. The new batch feature code is generated using a hash algorithm based on each batch of data. When different data exchange nodes use the same batch data and algorithm, the calculated batch signature code will be the same. The update module regenerates the corresponding batch data size Batch_n_Size_New for the received batch data Data_n. The new batch data size represents the data size, which is the number of data records or the file size. The update module updates the batch signature code Batch_n_Hash and the batch data size Batch_n_Size in the updated batch information to form new batch information (Batch_n_Code, Batch_n_Hash_New, Batch_n_Size_New, Batch_Table). The specific problems encountered during the data exchange and transmission process obtained from the verification batch information include: The data verification module receives and stores batch information sent by each data exchange node in real time. The data verification module uses a single batch number as a clue to cross-compare batch information with the same batch number sent by different data exchange nodes in order to discover that a single batch of exchanged data is lost or tampered with during the transmission between data exchange nodes. The data verification module uses the batch code table Batch_Table in the batch information as a clue to cross-compare the batch information split from the same original data sent by a data exchange node in order to determine whether the original data transmission is correct.

2. A data verification method applied in data exchange, based on the system described in claim 1, characterized in that, The steps are as follows: 1) Divide the exchanged data into batches in the initial data exchange node, and generate batch data and batch information; 2) Update the batch information of the current data exchange node; 3) Transmit the exchanged data and updated batch information to the next data exchange node; 4) Repeat steps 2) and 3) until the exchanged data is transmitted to the final data exchange node; 5) Verify all updated batch information and analyze the problems in the data exchange and transmission process.

3. The data verification method applied in data exchange according to claim 2, characterized in that, Step 1) specifically includes: 11) Set the exchange data Data to have been transmitted from the data exchange initiating system to the starting data exchange node Node_A, that is, Node_A already has the exchange data Data; 12) The batch partitioning module in the starting data exchange node Node_A divides the exchanged data into batches according to the network, database, and exchanged data, based on the amount of data that can be correctly transmitted at one time and the file size, to obtain batch data and batch information. The batch information includes batch code, batch feature code, batch data volume, and batch code table. 13) The batch segmentation module extracts each batch of data one by one, and uses each batch of data and its corresponding batch information to form an independent exchange data packet. The batch information includes the batch code, batch feature code, batch data volume and batch code table.

4. The data verification method applied in data exchange according to claim 3, characterized in that, Step 12) specifically includes: 121) The exchanged data is split into several small data packets, each data packet being a batch of data Data_1, Data_2, Data_3, ..., Data_N, and stored separately; 122) Generate corresponding batch codes Batch_1_Code, Batch_2_Code, Batch_3_Code, ..., Batch_N_Code for each batch of data Data_1, Data_2, Data_3, ..., Data_N; the batch codes are generated using a unique code generation algorithm based on the number of batches; different batches do not repeat the same code. 123) For each batch of data Data_1, Data_2, Data_3, ..., Data_N, generate corresponding batch feature codes Batch_1_Hash, Batch_2_Hash, Batch_3_Hash, ..., Batch_N_Hash; the batch feature codes are generated using a hash algorithm based on each batch of data; different data exchange nodes will calculate the same batch feature codes when using the same batch of data and hash algorithm; 124) For each batch of data Data_1, Data_2, Data_3, ..., Data_N, generate the corresponding batch data size Batch_1_Size, Batch_2_Size, Batch_3_Size, ..., Batch_N_Size; the batch data size represents the data size, which is the number of data records or the file size; 125) Arrange all batch codes in order according to the batch sequence (Batch_1_Code, Batch_2_Code, Batch_3_Code, ..., Batch_N_Code) to form a batch code table containing all batch codes.

5. The data verification method applied in data exchange according to claim 3, characterized in that, The batch division in step 12) is based on different original data types, as follows: 126) If the original data is a data table in a relational database, the batches should be divided according to the number of records in the data table; use a fixed number of records to divide the batches, and the number of records in each batch after division should be less than or equal to the fixed number of records; the fixed number of records should be modified according to the actual network architecture and transmission conditions; 127) If the original data is a large file, the batches should be divided according to the file storage size; use a fixed storage size to divide the batches, and the size of each batch of data after division should be less than or equal to the fixed storage size. 128) If the original data consists of multiple small files, batches should be divided by individual files; after division, each file constitutes a batch.

6. The data verification method applied in data exchange according to claim 3, characterized in that, The exchange data packets in step 13) are: (Data_1, Batch_1_Code, Batch_1_Hash, Batch_1_Size, Batch_Table), (Data_2, Batch_2_Code, Batch_2_Hash, Batch_2_Size, Batch_Table), (Data_3, Batch_3_Code, Batch_3_Hash, Batch_3_Size, Batch_Table), ..., (Data_N, Batch_N_Code, Batch_N_Hash, Batch_N_Size, Batch_Table); thus splitting the original exchange data Data into N independent small exchange data packets.

7. The data verification method applied in data exchange according to claim 2, characterized in that, Step 2) specifically includes: 21) The current data exchange node receives an exchange data packet (Data_n, Batch_n_Code, Batch_n_Hash, Batch_n_Size, Batch_Table) containing batch data and batch information, where n is 1, 2, 3, ..., N, and is any batched exchange data packet. The exchange data packet can come from this data exchange node or other data exchange nodes. 22) The update module determines whether the received batch data and batch information exchange data packets come from the current data exchange node; if they come from the current data exchange node, proceed to step 26); if they come from other data exchange nodes, proceed to step 23). 23) The update module regenerates the corresponding batch feature code Batch_n_Hash_New for the received batch data Data_n; the new batch feature code is generated using a hash algorithm based on each batch of data; different data exchange nodes will calculate the same batch feature code when using the same batch data and algorithm; 24) The update module regenerates the corresponding batch data size Batch_n_Size_New for the received batch data Data_n; the new batch data size represents the data size, which is the number of data records or the file size; 25) The update module updates the batch feature code Batch_n_Hash and batch data size Batch_n_Size in the batch information to form new batch information (Batch_n_Code, Batch_n_Hash_New, Batch_n_Size_New, Batch_Table). 26) The update module combines the node number and the updated batch information into an exchange data packet (Node_n, Batch_n_Code, Batch_n_Hash, Batch_n_Size, Batch_Table) and sends it to the data verification module.

8. The data verification method applied in data exchange according to claim 2, characterized in that, Step 3) specifically includes: The current data exchange node's transmission module combines the exchange data, node number, and batch information into an exchange data packet (Data_n, Node_n, Batch_n_Code, Batch_n_Hash, Batch_n_Size, Batch_Table) and sends it to the next data exchange node as needed for data transmission. The data transmission module uses different implementation methods depending on the different batch data types: 311) If the batch data type is a relational database, the data transfer module uses a data extraction ETL tool, a database job, or an extraction program. 312) If the batch data type is a file, the data transfer module uses a message queue, FTP tool, operating system transfer job, or control transfer program.

9. The data verification method applied in data exchange according to claim 2, characterized in that, Step 4) specifically includes: 41) Repeat steps 2) and 3) until the exchange data packet, which combines the exchanged data and batch information, is delivered to the final data exchange node; 42) If all batches of exchanged data packets are transmitted correctly, the final data exchange node will receive an exchanged data packet consisting of all the exchanged data from batches 1, 2, 3, ..., N, and the updated batch information. 43) The update module of the final data exchange node uses the batch number in the updated batch information to determine whether the data transmission is correct; if the data transmission is correct, proceed to step 44); if the data transmission is incorrect, stop running and issue a notification. 44) The batch division module of the final data exchange node assembles each batch of data according to the batch number sequence and batch data in the batch code table Batch_Table in any batch information, and restores the original data before the batch split in step 1), so as to complete the transfer of the original data from the initial data exchange node Node_A to the final data exchange node Node_F.

10. The data verification method applied in data exchange according to claim 2, characterized in that, Step 5) specifically includes: 51) The data verification module receives and stores batch information sent by each data exchange node in real time; 52) The data verification module uses a single batch number as a clue to cross-compare batch information with the same batch number sent by different data exchange nodes in order to discover that single batch exchange data is lost or tampered with during the transmission between data exchange nodes. 53) The data verification module uses the batch code table Batch_Table in the batch information as a clue to cross-compare the batch information split from the same original data sent by a data exchange node to determine whether the original data transmission is correct.

Images