A data comparison and reconciliation method and system in a cross-network data exchange and transmission scenario

Abstract

The present invention proposes a data comparison and reconciliation method and system in a cross-network data exchange and transmission scenario, which relates to the field of big data technology. The front-end data reconciliation system receives business data, generates verification information including a front-end verification code, business data format, data volume and a unique identification code, and sends it to the rear-end data reconciliation system through a unidirectional transmission channel after secondary packaging. The rear-end system decapsulates and generates corresponding verification information, compares it with the received information, and marks the verification status according to the result. If the verification fails, the abnormal classification and traceability mechanism is triggered, and automatic retransmission or manual intervention is performed. Among them, the verification code generation model based on the multimodal adversarial hash network can extract multimodal features, and dynamically select the hash path to generate the verification code according to the information entropy, thereby improving the accuracy and speed of multimodal data verification, and the abnormal classification covers content tampering, protocol anomalies, etc., and the transaction records can be found in the alliance chain according to the transmission sequence number during tracing, so as to improve the problem location and problem solving capabilities.

Description

Technical Field

[0001] The present invention relates to a data comparison and reconciliation method and system in a cross-network data exchange and transmission scenario, belonging to the technical field of big data. Background Art

[0002] In today's digital age, cross-system data exchange and transmission are becoming increasingly frequent, especially in industries such as finance and e-commerce. A large amount of data interaction and verification work is required between different systems to ensure data accuracy and consistency. Data comparison and reconciliation, as a key link in ensuring data quality, plays a vital role in cross-network data exchange and transmission scenarios.

[0003] For example, the Chinese invention patent "Hash-based reconciliation method and device" with announcement number CN112785408B discloses first obtaining the transaction data to be checked, calculating the hash value corresponding to each transaction data and generating the leaf nodes of the Merkle tree based on these hash values, and then generating multiple reconciliation files based on the leaf nodes of the Merkle tree, and finally reconciling based on the multiple reconciliation files generated by the upstream system and the downstream system. In addition, the method will also store the hash values ​​corresponding to the transaction data in each partition of the target hash ring, and cache the target hash ring in the cache server to improve the efficiency of data retrieval and processing. At the same time, the size of the hash ring and the length of the cache chain can be dynamically adjusted according to actual conditions.

[0004] However, the hash-based reconciliation method proposed in the above invention mainly relies on traditional technologies such as Merkle trees and hash rings, and only performs simple hash mapping on specific primary keys of transaction data, without deeply mining the characteristic information inside the data. It is insufficient to process multimodal business data including text, tables, and images, and is unable to fully extract the characteristics of different types of data, resulting in low verification accuracy and efficiency in complex data scenarios. On the other hand, during the data transmission process, the invention lacks an effective abnormal classification and traceability mechanism. When problems such as data tampering, loss, and protocol anomalies occur, it is difficult to quickly and accurately locate the root cause of the problem, which is not conducive to taking effective solutions in a timely manner.

[0005] Therefore, there is an urgent need for a data comparison and reconciliation method that can handle complex data, improve verification accuracy and speed, and solve problems in a timely manner. Summary of the invention

[0006] In order to solve the above problems existing in the prior art, the present invention proposes a data comparison and reconciliation method and system in a cross-network data exchange and transmission scenario.

[0007] The technical solution of the present invention is as follows:

[0008] On the one hand, the present invention provides a data comparison and reconciliation method in a cross-network data exchange and transmission scenario, the method comprising:

[0009] The front-end data reconciliation system connects to the front-end business system and receives business data through the business interface;

[0010] Based on the business data, verification information including a pre-check code, a pre-business data format, a pre-business data volume and a unique identification code is generated, and after the verification information is attached with a transmission sequence number, the business data is re-encapsulated to obtain a re-encapsulated data packet, and sent to a post-data reconciliation system through a unidirectional transmission channel;

[0011] The post-data reconciliation system receives the secondary encapsulated data packet and decapsulates it, and generates corresponding verification information based on the business data obtained by decapsulation, including the post-check code, the post-business data format and the post-business data volume; the verification information generated by the post-data reconciliation system is compared with the received verification information to obtain the comparison result, and the verification status is marked according to the comparison result. If the verification fails, the abnormal classification and traceability mechanism is triggered, and automatic retransmission or manual intervention is performed.

[0012] As a preferred embodiment of the present invention, generating a check code based on business data specifically comprises constructing a check code generation model of the same structure in a front-end data reconciliation system and a back-end data reconciliation system, wherein the check code generation model extracts text, table and image features of business data based on a multimodal adversarial hash network, and dynamically selects a path to generate a check code according to information entropy, the front-end data reconciliation system generates a front check code, and the back-end data reconciliation system generates a back check code, wherein:

[0013] The multimodal adversarial hashing network includes a branch network, an attention fusion layer and a binarization layer. Specifically:

[0014] The branch network includes a text branch layer, a table branch layer and an image branch layer. The text branch layer extracts text features through a lightweight Transformer architecture, the table branch layer outputs table features by modeling table row-column relationships through a graph attention network GAT, and the image branch layer extracts image features through a depth-separable convolution.

[0015] The attention fusion layer is used to fuse text features, table features and image features, which can be expressed as:

[0016] ;

[0017] ;

[0018] ;

[0019] In the formula, For fusion features; For the The attention weights of the branches; For the The characteristics of each branch; is the text feature; is the table feature; is the image feature; is the index of the branch in the branch network; is the attention weight matrix in the attention fusion layer;

[0020] The fused features are input into the binary layer of adversarial training to generate an intermediate check code, which can be expressed as:

[0021] ;

[0022] ;

[0023] In the formula, is the intermediate check code; is the hash layer weight matrix; is the bias term of the binarization layer.

[0024] As a preferred embodiment of the present invention, dynamically selecting a path according to information entropy is specifically to dynamically select a compression or block path in a multi-branch path tree according to the information entropy of the intermediate check code to generate a final check code, which is expressed as:

[0025] ;

[0026] In the formula, is the final check code. If the final check code is the pre-check code, it is recorded as , if the final check code is a post-check code, it is recorded as ; is the Sigmoid function; is the output layer attention weight; To perform splicing operations; Output for XOR compression path; Output for the block hash path; Select the corresponding bias item for the path;

[0027] in:

[0028] The dynamic path selection rule of the multi-branch path tree is specifically as follows:

[0029] When the information entropy of the intermediate check code is less than the preset entropy threshold, the XOR compression path is selected. The XOR compression path is expressed as:

[0030] ;

[0031] In the formula, is the length of the intermediate check code; To perform a bitwise XOR operation; For the Bit middle check code; For the Bit middle check code; is the position index of the intermediate check code;

[0032] When the information entropy of the intermediate check code is greater than or equal to the preset entropy threshold, the block hash path is selected. The block hash path is expressed as:

[0033] ;

[0034] In the formula, is the number of blocks; It is a lightweight convolution operation; For the The intermediate checksum of each block; The block index.

[0035] As a preferred implementation of the present invention, the method further includes performing cross-network collaborative training on the constructed multimodal adversarial hash network, specifically:

[0036] The front-end data reconciliation system uses a homomorphic encryption algorithm to encrypt and transmit the gradient, and updates the gradient of the multimodal adversarial hash network in the back-end data reconciliation system, which is expressed as:

[0037] ;

[0038] In the formula, is the learning rate; Updated gradients in multimodal adversarial hashing networks in post-data reconciliation systems; Gradients before updating in multimodal adversarial hashing networks in post-data reconciliation systems; To perform decryption operations; The gradient is encrypted by the homomorphic encryption algorithm.

[0039] As a preferred implementation of the present invention, the abnormal classification triggered by the verification failure is specifically as follows:

[0040] If the comparison result is that the pre-check code and the post-check code are inconsistent, the anomaly is marked as content tampering; if the comparison result is that the pre-business data format and the post-business data format are inconsistent, the anomaly is marked as protocol anomaly; if the comparison result is that the pre-business data volume and the post-business data volume are inconsistent, the anomaly is marked as data corruption; if the unique identification code is missing, the anomaly is marked as data loss.

[0041] As a preferred implementation of the present invention, the method further includes packaging the pre-check code, the transmission sequence number and the corresponding timestamp into a blockchain transaction, specifically:

[0042] Define the blockchain transaction data set as , where each transaction Contains the prefix check code , transmission sequence number and the corresponding timestamp ,Right now ; The node set corresponding to the alliance chain is recorded as ;

[0043] When a new transaction is generated, it is broadcast to each node of the alliance chain for verification. For the verification function, for the transaction and nodes , through a validation function that outputs a Boolean value Indicates the node's verification result of the transaction. If the verification is successful, it returns True, otherwise it returns False;

[0044] Define the consensus threshold as , when satisfied When trading It is considered to have passed the consensus verification and is written into the alliance chain;

[0045] definition For the blockchain of the alliance chain, when the transaction After passing consensus verification, it is added to the current block to be packaged In the updated block .

[0046] As a preferred implementation of the present invention, the traceability mechanism triggered by verification failure is specifically:

[0047] The post-data reconciliation system checks the transmission sequence number involved when the verification fails. Search for the corresponding transaction record in the alliance chain. The search function is , the return value is a transaction set containing the transmission sequence number , expressed as:

[0048] ;

[0049] For transaction collection Every transaction in , extract the corresponding pre-check code and timestamp, compare the corresponding pre-check code and timestamp with the post-check code generated by the post-data reconciliation system to determine whether the business data has been tampered with or lost during the transmission process.

[0050] On the other hand, the present invention also provides a data comparison and reconciliation system in a cross-network data exchange and transmission scenario, the system comprising a front-end data reconciliation system deployed in a front-end network and a rear-end data reconciliation system deployed in a rear-end network, and data transmission is performed through a unidirectional transmission channel constructed by the data exchange system, wherein:

[0051] The front-end data reconciliation system is connected to the front-end business system, receives business data through the business interface, and generates verification information including a front-end verification code, a front-end business data format, a front-end business data volume and a unique identification code based on the business data, and after attaching a transmission sequence number to the verification information, re-encapsulates the business data to obtain a re-encapsulated data packet, and sends it to the rear-end data reconciliation system through the unidirectional transmission channel;

[0052] The post-data reconciliation system receives the secondary encapsulated data packet and decapsulates it, and generates corresponding verification information based on the business data obtained by decapsulation, including a post-check code, a post-business data format, and a post-business data volume; obtains a comparison result by comparing the corresponding verification information with the received verification information, marks the verification status according to the comparison result, and triggers an abnormal classification and tracing mechanism if the verification fails, and performs automatic retransmission or manual intervention;

[0053] Among them, generating a verification code based on business data specifically involves building a verification code generation model with the same structure in the front-end data reconciliation system and the back-end data reconciliation system. The verification code generation model extracts text, table and image features of business data based on a multimodal adversarial hash network, and dynamically selects a path to generate a verification code according to information entropy. The front-end data reconciliation system generates a front verification code, and the back-end data reconciliation system generates a back verification code, wherein:

[0054] The multimodal adversarial hashing network includes a branch network, an attention fusion layer and a binarization layer. Specifically:

[0055] The branch network includes a text branch layer, a table branch layer and an image branch layer. The text branch layer extracts text features through a lightweight Transformer architecture, the table branch layer outputs table features by modeling table row-column relationships through a graph attention network GAT, and the image branch layer extracts image features through a depth-separable convolution.

[0056] The attention fusion layer is used to fuse text features, table features and image features, which can be expressed as:

[0057] ;

[0058] ;

[0059] ;

[0060] In the formula, For fusion features; For the The attention weights of the branches; For the The characteristics of each branch; is the text feature; is the table feature; is the image feature; is the index of the branch in the branch network; is the attention weight matrix in the attention fusion layer;

[0061] The fused features are input into the binary layer of adversarial training to generate an intermediate check code, which can be expressed as:

[0062] ;

[0063] ;

[0064] In the formula, is the intermediate check code; is the hash layer weight matrix; is the bias term of the binarization layer;

[0065] Dynamically selecting a path based on information entropy is specifically to dynamically select a compression or block path in a multi-branch path tree based on the information entropy of the intermediate check code to generate the final check code, which is expressed as:

[0066] ;

[0067] In the formula, is the final check code. If the final check code is the pre-check code, it is recorded as , if the final check code is a post-check code, it is recorded as ; is the Sigmoid function; is the output layer attention weight; To perform splicing operations; Output for XOR compression path; Output for the block hash path; Select the corresponding bias item for the path;

[0068] in:

[0069] The dynamic path selection rule of the multi-branch path tree is specifically as follows:

[0070] When the information entropy of the intermediate check code is less than the preset entropy threshold, the XOR compression path is selected. The XOR compression path is expressed as:

[0071] ;

[0072] In the formula, is the length of the intermediate check code; To perform a bitwise XOR operation; For the Bit middle check code; For the Bit middle check code; is the position index of the intermediate check code;

[0073] When the information entropy of the intermediate check code is greater than or equal to the preset entropy threshold, the block hash path is selected. The block hash path is expressed as:

[0074] ;

[0075] In the formula, is the number of blocks; It is a lightweight convolution operation; For the The intermediate checksum of each block; The block index.

[0076] On the other hand, the present embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a data comparison and reconciliation method in a cross-network data exchange and transmission scenario as described in any embodiment of the present invention is implemented.

[0077] On the other hand, the present embodiment further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a data comparison and reconciliation method in a cross-network data exchange and transmission scenario as described in any embodiment of the present invention.

[0078] The present invention has the following beneficial effects:

[0079] 1. The present invention provides a data comparison and reconciliation method and system in a cross-network data exchange and transmission scenario. A verification code generation model is constructed. Multimodal features are extracted through a branch network, and dynamic weighted fusion is combined with an attention mechanism to improve the verification code generation accuracy. The verification code generation model dynamically selects XOR compression or block hash path based on the information entropy threshold, taking into account verification efficiency and security. The verification code generation model also uses homomorphic encryption to transmit gradients to achieve secure updates of model parameters and avoid data leakage.

[0080] 2. The present invention is a data comparison and reconciliation method and system in a cross-network data exchange and transmission scenario. The verification code, transmission sequence number and timestamp are written into the alliance chain, and the consensus mechanism is used to ensure that the data cannot be tampered with. In combination with abnormal classification rules (content tampering, protocol abnormality, data corruption, data loss), automatic traceability is achieved, and problems are located and solved in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0082] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0083] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0084] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0085] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0086] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0087] Embodiment 1:

[0088] See also Figure 1This embodiment provides a data comparison and reconciliation method in a cross-network data exchange and transmission scenario, including the following steps:

[0089] S1. In scenarios with extremely high requirements for data security, in order to ensure data consistency between the front and back networks and prevent any unauthorized two-way data flow, corresponding data reconciliation systems are deployed in the physically isolated front network and back network, namely the front data reconciliation system and the back data reconciliation system, and a one-way transmission channel is built through the data exchange system to protect the security and integrity of the data;

[0090] S2, the front-end data reconciliation system connects to the front-end business system, receives business data through the business interface, and the business interface includes ftp, myslq and http interfaces; generates verification information including a front-end verification code, a front-end business data format, a front-end business data volume and a unique identification code based on the business data, and after adding a transmission sequence number to the verification information, re-encapsulates the business data to obtain a re-encapsulated data packet, and sends it to the rear-end data reconciliation system through the unidirectional transmission channel, wherein the re-encapsulated data packet = {business data, front-end verification code, front-end business data format, front-end business data volume, unique identification code, transmission sequence number};

[0091] S21. The pre-data reconciliation system generates verification information including a pre-verification code, a pre-business data format, a pre-business data volume and a unique identification code based on the business data, specifically:

[0092] Construct a verification code generation model, which extracts text, table, and image features of business data based on a multimodal adversarial hash network, and dynamically selects a path to generate a verification code according to information entropy. Preferably, the verification code generation model is deployed in both the front-end and the back-end data reconciliation systems. In the front-end data reconciliation system, it is used to generate a front verification code and send it to the back-end data reconciliation system through a one-way transmission channel. The verification code generation model in the back-end data reconciliation system is used to generate a back verification code and compare it with the front verification code to verify the integrity and consistency of the business data during transmission;

[0093] The one-way transmission channel is preset with a transmission format. By checking the format of the business data (such as JSON, CSV, PNG), it is ensured that the business data conforms to the preset structure of the one-way transmission channel, and the format of the business data is recorded as the pre-set business format;

[0094] Calculate the size (ie, number of bytes) of the service data as the amount of pre-service data, where the pre-service data is used to detect possible loss or damage during data transmission;

[0095] A unique identification code is generated for each data packet to be re-encapsulated and a transmission sequence number is assigned. In this embodiment, a UUID algorithm is used to generate a unique identification code. Since the UUID algorithm is a common algorithm in the field, it is not described here. The unique identification code is used to ensure the uniqueness of the data packet; the transmission sequence number is an incremental serial number, which is used to help the post-data reconciliation system confirm the order of the data packets;

[0096] This embodiment solves the problem of packet loss and disorder in cross-network transmission by using the transmission sequence number used to track the transmission sequence mark in the secondary encapsulation;

[0097] S211. The constructed multimodal adversarial hash network includes a branch network, an attention fusion layer and a binarization layer. Specifically:

[0098] S2111, the branch network includes a text branch layer, a table branch layer and an image branch layer, the text branch layer extracts text features through a lightweight Transformer architecture, the table branch layer outputs table features by modeling table row-column relationships through a graph attention network GAT, and the image branch layer extracts image features through a depthwise separable convolution; the lightweight Transformer architecture, graph attention network GAT and depthwise separable convolution in the branch network are all common networks in the art and will not be described in detail here;

[0099] S2112, the attention fusion layer is used to fuse text features, table features and image features, which can be expressed as:

[0100] ;

[0101] ;

[0102] ;

[0103] In the formula, For fusion features; For the The attention weights of the branches; For the The characteristics of each branch; is the text feature; is the table feature; is the image feature; is the index of the branch in the branch network; is the learnable attention weight matrix in the attention fusion layer;

[0104] S2113, input the fusion feature into the binary layer of adversarial training to generate an intermediate check code, which is expressed as:

[0105] ;

[0106] ;

[0107] In the formula, is the intermediate check code; is the hash layer weight matrix; is the bias term of the binarization layer;

[0108] S2114, dynamically selecting a path based on information entropy is specifically to dynamically select a compression or block path in a multi-branch path tree according to the information entropy of the intermediate check code to generate a final check code, which is expressed as:

[0109] ;

[0110] In the formula, is the final verification code. Since both the pre-data reconciliation system and the post-data reconciliation system are equipped with verification code generation models, in order to distinguish the generated verification codes, if the final verification code is the pre-verification code, it is recorded as , if the final check code is a post-check code, it is recorded as ; is the Sigmoid function; is the output layer attention weight; To perform splicing operations; Output for XOR compression path; Output for the block hash path; Select the corresponding bias item for the path;

[0111] in:

[0112] The dynamic path selection rule of the multi-branch path tree is specifically as follows:

[0113] When the information entropy of the intermediate check code is less than the preset entropy threshold, the XOR compression path is selected. The XOR compression path is expressed as:

[0114] ;

[0115] In the formula, is the length of the intermediate check code; To perform a bitwise XOR operation; For the Bit middle check code; For the Bit middle check code; is the position index of the intermediate check code;

[0116] When the information entropy of the intermediate check code is greater than or equal to the preset entropy threshold, the block hash path is selected. The block hash path is expressed as:

[0117] ;

[0118] In the formula, is the number of blocks; It is a lightweight convolution operation; For the The intermediate checksum of each block; is the block index;

[0119] S215. The method further includes performing cross-network collaborative training on the constructed multimodal adversarial hash network, and introducing a neural network architecture search NAS during the training process to dynamically optimize the branch network structure, wherein:

[0120] The cross-network collaborative training is specifically as follows:

[0121] The front-end data reconciliation system uses a homomorphic encryption algorithm to encrypt and transmit the gradient, and updates the gradient of the multimodal adversarial hash network in the back-end data reconciliation system, which is expressed as:

[0122] ;

[0123] In the formula, is the learning rate; Updated gradients in multimodal adversarial hashing networks in post-data reconciliation systems; Gradients before updating in multimodal adversarial hashing networks in post-data reconciliation systems; To perform decryption operations; The gradient encrypted by the homomorphic encryption algorithm;

[0124] Through cross-network collaborative training, the problem of unbalanced model training data under physical isolation is solved, and the generalization ability of adversarial samples is improved; since the homomorphic encryption algorithm is a common encryption algorithm in this field, the specific encryption and decryption process will not be described here;

[0125] The neural network architecture search NAS is introduced in the training process to dynamically optimize the branch network structure, specifically:

[0126] The search space is defined to include the number of Transformer layers, the number of GAT heads, and the size of the convolution kernel. In this embodiment, the number of lightweight Transformer layers and heads used in the text branch layer of the multimodal adversarial hash network are set to 4, the number of graph attention network GAT heads used in the table branch layer is 2, and the convolution kernel size corresponding to the depthwise separable convolution used in the image branch layer is 3×3; then in the defined search space, the more layers the model has, the more complex text feature relationships it can theoretically learn, but it will also increase the amount of calculation and training time, and the number of heads determines the number of attention calculations performed in parallel by the model when calculating the attention weights. Different convolution kernel sizes can capture feature patterns of different sizes in the image, for example, a small convolution kernel is suitable for capturing local details, and a large convolution kernel can capture more global features. Therefore, in this embodiment, the number of Transformer layers is set to 1-6 layers, the number of GAT heads is 1-4 heads, and the convolution kernel size is 3×3 to 7×7, and the optimal parameter combination is determined by a search range larger than the initial setting value;

[0127] The reinforcement learning agent selects the optimal branch structure combination to minimize the adversarial training loss Computational overhead The weighted sum is the objective function, which can be expressed as:

[0128] ;

[0129] In the formula, To calculate the cost weight; is the model floating point operation amount;

[0130] Apply the searched optimal structure to the pre- and post-data reconciliation systems;

[0131] S22. The method further includes packaging the pre-check code, the transmission sequence number and the corresponding timestamp into a blockchain transaction, verifying the data integrity based on the smart contract, and writing the transaction into the alliance chain. Specifically:

[0132] Define the blockchain transaction data set as , where each transaction Contains the prefix check code , transmission sequence number and the corresponding timestamp ,Right now ;

[0133] The alliance chain is composed of multiple nodes, and the node set is recorded as ;

[0134] When a new transaction is generated, it needs to be broadcast to each node of the alliance chain for verification. For the verification function, for the transaction and nodes , verification function The output result is a Boolean value, which indicates the node's verification result of the transaction. If the verification is successful, True is returned, otherwise False is returned.

[0135] In the consortium chain, in order to ensure the validity of the transaction, a certain number of nodes are required to reach a consensus, and the consensus threshold is defined as , then when it satisfies When trading It is considered to have passed consensus verification and can be written into the consortium chain;

[0136] definition For the blockchain of the alliance chain, each block Contains multiple transactions. When the transaction After passing consensus verification, it is added to the current block to be packaged In the updated block ;

[0137] S3. The post-data reconciliation system receives the secondary encapsulated data packet and decapsulates it, and generates corresponding verification information based on the business data obtained by decapsulation, including the post-check code, the post-business data format, and the post-business data volume; compares the verification information generated by the post-data reconciliation system with the received verification information to obtain a comparison result, marks the verification status according to the comparison result, and triggers the abnormal classification and tracing mechanism if the verification fails, and performs automatic retransmission or manual intervention;

[0138] S31, the comparison includes comparing the pre-check code with the post-check code, comparing the pre-service data format with the post-service data format, and comparing the pre-service data volume with the post-service data volume. If the comparison results are consistent, the verification state is marked as verification success; if the comparison results are inconsistent, the verification state is marked as verification failure;

[0139] S32. If the verification fails, the abnormal classification and tracing mechanism is triggered, where:

[0140] S321, the abnormal classification is specifically:

[0141] If the comparison result shows that the pre-check code and post-check code are inconsistent, the abnormality is marked as content tampering, and the data encapsulation module or transmission intermediate node is traced back;

[0142] If the comparison result shows that the format of the pre-service data is inconsistent with that of the post-service data, the exception is marked as a protocol exception and the compliance of the pre-service data reconciliation network interface is checked;

[0143] If the comparison result shows that the amount of pre-service data is inconsistent with the amount of post-service data, the anomaly is marked as data corruption to verify the stability of the one-way transmission channel;

[0144] If the unique identifier is missing, mark the anomaly as data loss and check firewall policies or network interruptions;

[0145] S322. The traceability mechanism is to obtain the stored evidence data from the blockchain for audit tracing when the post-data reconciliation system fails to verify. Specifically:

[0146] The post-data reconciliation system checks the transmission sequence number involved when the verification fails. Search for the corresponding transaction record in the alliance chain. The search function is , the return value is a transaction set containing the transmission sequence number , expressed as:

[0147] ;

[0148] For transaction collection Every transaction in , extract the corresponding front check code and timestamp, and compare this information with the relevant data in the post-data reconciliation system to determine whether the data has been tampered with or lost during the transmission process. If the data is found to be abnormal, further analyze the timestamp to locate the time point when the problem may occur, so as to conduct audit tracing and find out the root cause of the problem, such as whether there is a failure in the data generation link, data transmission link or data receiving link of the post-data reconciliation system.

[0149] By recording the pre-check code, transmission sequence number and timestamp into the alliance chain, and utilizing the tamper-proof and distributed storage characteristics of the blockchain, a reliable audit and traceability mechanism is provided for the integrity and transmission process of the data, thus enhancing the security and reliability of the entire data reconciliation system.

[0150] S33, the automatic retransmission or manual intervention is specifically:

[0151] The service data that failed verification is added to the retransmission queue and retried according to the exponential backoff strategy (such as 1s, 2s, 4s intervals) for a maximum of 3 times;

[0152] When continuous verification fails, an alarm is triggered to notify the operation and maintenance personnel, and the abnormal log is recorded in the blockchain evidence storage system;

[0153] This embodiment combines automatic retransmission with manual backup to ensure eventual consistency of services.

[0154] Embodiment 2:

[0155] This embodiment provides a data comparison and reconciliation system in a cross-network data exchange and transmission scenario, wherein the system includes a front-end data reconciliation system deployed in a front-end network and a rear-end data reconciliation system deployed in a rear-end network, and data is transmitted through a unidirectional transmission channel constructed by the data exchange system, wherein:

[0156] The front-end data reconciliation system is connected to the front-end business system, receives business data through the business interface, and generates verification information including a front-end verification code, a front-end business data format, a front-end business data volume and a unique identification code based on the business data, and after attaching a transmission sequence number to the verification information, re-encapsulates the business data to obtain a re-encapsulated data packet, and sends it to the rear-end data reconciliation system through the unidirectional transmission channel;

[0157] The post-data reconciliation system receives the secondary encapsulated data packet and decapsulates it, and generates corresponding verification information based on the business data obtained by decapsulation, including a post-check code, a post-business data format, and a post-business data volume; obtains a comparison result by comparing the corresponding verification information with the received verification information, marks the verification status according to the comparison result, and triggers an abnormal classification and tracing mechanism if the verification fails, and performs automatic retransmission or manual intervention;

[0158] Among them, generating a verification code based on business data specifically involves building a verification code generation model with the same structure in the front-end data reconciliation system and the back-end data reconciliation system. The verification code generation model extracts text, table and image features of business data based on a multimodal adversarial hash network, and dynamically selects a path to generate a verification code according to information entropy. The front-end data reconciliation system generates a front verification code, and the back-end data reconciliation system generates a back verification code, wherein:

[0159] The multimodal adversarial hashing network includes a branch network, an attention fusion layer and a binarization layer. Specifically:

[0160] The branch network includes a text branch layer, a table branch layer and an image branch layer. The text branch layer extracts text features through a lightweight Transformer architecture, the table branch layer outputs table features by modeling table row-column relationships through a graph attention network GAT, and the image branch layer extracts image features through a depth-separable convolution.

[0161] The attention fusion layer is used to fuse text features, table features and image features, which can be expressed as:

[0162] ;

[0163] ;

[0164] ;

[0165] In the formula, For fusion features; For the The attention weights of the branches; For the The characteristics of each branch; is the text feature; is the table feature; is the image feature; is the index of the branch in the branch network; is the attention weight matrix in the attention fusion layer;

[0166] The fused features are input into the binary layer of adversarial training to generate an intermediate check code, which can be expressed as:

[0167] ;

[0168] ;

[0169] In the formula, is the intermediate check code; is the hash layer weight matrix; is the bias term of the binarization layer;

[0170] Dynamically selecting a path based on information entropy is specifically to dynamically select a compression or block path in a multi-branch path tree based on the information entropy of the intermediate check code to generate the final check code, which is expressed as:

[0171] ;

[0172] In the formula, is the final check code. If the final check code is the pre-check code, it is recorded as , if the final check code is a post-check code, it is recorded as ; is the Sigmoid function; is the output layer attention weight; To perform splicing operations; Output for XOR compression path; Output for the block hash path; Select the corresponding bias item for the path;

[0173] in:

[0174] The dynamic path selection rule of the multi-branch path tree is specifically as follows:

[0175] When the information entropy of the intermediate check code is less than the preset entropy threshold, the XOR compression path is selected. The XOR compression path is expressed as:

[0176] ;

[0177] In the formula, is the length of the intermediate check code; To perform a bitwise XOR operation; For the Bit middle check code; For the Bit middle check code; is the position index of the intermediate check code;

[0178] When the information entropy of the intermediate check code is greater than or equal to the preset entropy threshold, the block hash path is selected. The block hash path is expressed as:

[0179] ;

[0180] In the formula, is the number of blocks; It is a lightweight convolution operation; For the The intermediate checksum of each block; The block index.

[0181] Embodiment 3:

[0182] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, a data comparison and reconciliation method in a cross-network data exchange and transmission scenario as described in any embodiment of the present invention is implemented.

[0183] Embodiment 4:

[0184] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, a data comparison and reconciliation method in a cross-network data exchange and transmission scenario as described in any embodiment of the present invention is implemented.

[0185] It is worth noting that the system, electronic device and computer-readable storage medium described in the present invention are all based on the same inventive concept as the method described in Example 1 of the present invention, and will not be described in detail here.

[0186] In the embodiments of the present invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. A and B may be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c may represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c may be single or multiple.

[0187] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0188] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0189] In several embodiments provided by the present invention, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk and other media that can store program codes.

[0190] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A data comparison and reconciliation method in a cross-network data exchange and transmission scenario, characterized in that: The method comprises: The front-end data reconciliation system is connected to the front-end business system and receives business data through the business interface; Based on the business data, verification information including a pre-check code, a pre-business data format, a pre-business data volume and a unique identification code is generated, and after the verification information is attached with a transmission sequence number, the business data is re-encapsulated to obtain a re-encapsulated data packet, and sent to a post-data reconciliation system through a unidirectional transmission channel; The post-data reconciliation system receives the secondary encapsulated data packet and decapsulates it, and generates corresponding verification information based on the business data obtained by decapsulation, including the post-check code, the post-business data format, and the post-business data volume; compares the verification information generated by the post-data reconciliation system with the received verification information to obtain the comparison result, marks the verification status according to the comparison result, and triggers the abnormal classification and tracing mechanism if the verification fails, and performs automatic retransmission or manual intervention; Among them, generating a verification code based on business data specifically involves building a verification code generation model with the same structure in the front-end data reconciliation system and the back-end data reconciliation system. The verification code generation model extracts text, table and image features of business data based on a multimodal adversarial hash network, and dynamically selects a path to generate a verification code according to information entropy. The front-end data reconciliation system generates a front verification code, and the back-end data reconciliation system generates a back verification code, wherein: The multimodal adversarial hashing network includes a branch network, an attention fusion layer and a binarization layer. Specifically: The branch network includes a text branch layer, a table branch layer and an image branch layer. The text branch layer extracts text features through a lightweight Transformer architecture, the table branch layer outputs table features by modeling table row-column relationships through a graph attention network GAT, and the image branch layer extracts image features through a depth-separable convolution. The attention fusion layer is used to fuse text features, table features and image features, which can be expressed as: ； ； ； In the formula, For fusion features; For the The attention weights of the branches; For the The characteristics of each branch; is the text feature; is the table feature; is the image feature; is the index of the branch in the branch network; is the attention weight matrix in the attention fusion layer; The fused features are input into the binary layer of adversarial training to generate an intermediate check code, which can be expressed as: ； ； In the formula, is the intermediate check code; is the hash layer weight matrix; is the bias term of the binarization layer; Dynamically selecting a path based on information entropy is specifically to dynamically select a compression or block path in a multi-branch path tree based on the information entropy of the intermediate check code to generate the final check code, which is expressed as: ； In the formula, is the final check code. If the final check code is the pre-check code, it is recorded as , if the final check code is a post-check code, it is recorded as ; is the Sigmoid function; is the output layer attention weight; To perform splicing operations; Output for XOR compression path; Output for the block hash path; Select the corresponding bias item for the path; in: The dynamic path selection rule of the multi-branch path tree is specifically: When the information entropy of the intermediate check code is less than the preset entropy threshold, the XOR compression path is selected. The XOR compression path is expressed as: ； In the formula, is the length of the intermediate check code; To perform a bitwise XOR operation; For the Bit middle check code; For the Bit middle check code; is the position index of the intermediate check code; When the information entropy of the intermediate check code is greater than or equal to the preset entropy threshold, the block hash path is selected. The block hash path is expressed as: ； In the formula, is the number of blocks; It is a lightweight convolution operation; For the The intermediate checksum of each block; The block index.

2. According to a method for comparing and reconciling data in a cross-network data exchange and transmission scenario according to claim 1, it is characterized in that: The method also includes performing cross-network collaborative training on the constructed multimodal adversarial hash network, specifically: The front-end data reconciliation system uses a homomorphic encryption algorithm to encrypt and transmit the gradient, and updates the gradient of the multimodal adversarial hash network in the back-end data reconciliation system, which is expressed as: ； In the formula, is the learning rate; Updated gradients in multimodal adversarial hashing networks in post-data reconciliation systems; Gradients before updating in multimodal adversarial hashing networks in post-data reconciliation systems; To perform decryption operations; The gradient is encrypted by the homomorphic encryption algorithm.

3. According to a method for comparing and reconciling data in a cross-network data exchange and transmission scenario according to claim 2, it is characterized in that: The specific exception categories triggered by verification failure are: If the comparison result is that the pre-check code and the post-check code are inconsistent, the anomaly is marked as content tampering; if the comparison result is that the pre-business data format and the post-business data format are inconsistent, the anomaly is marked as protocol anomaly; if the comparison result is that the pre-business data volume and the post-business data volume are inconsistent, the anomaly is marked as data corruption; if the unique identification code is missing, the anomaly is marked as data loss.

4. According to claim 3, a data comparison and reconciliation method in a cross-network data exchange and transmission scenario is characterized in that: The method further includes packaging the pre-check code, the transmission sequence number and the corresponding timestamp into a blockchain transaction, specifically: Define the blockchain transaction data set as , where each transaction Contains the prefix check code , transmission sequence number and the corresponding timestamp ,Right now ; The node set corresponding to the alliance chain is recorded as ; When a new transaction is generated, it is broadcast to each node of the alliance chain for verification. For the verification function, for the transaction and nodes , through a validation function that outputs a Boolean value Indicates the node's verification result of the transaction. If the verification is successful, it returns True, otherwise it returns False; Define the consensus threshold as , when satisfied When trading It is considered to have passed the consensus verification and is written into the alliance chain; definition For the blockchain of the alliance chain, when the transaction After passing consensus verification, it is added to the current block to be packaged In the updated block .

5. According to claim 4, a data comparison and reconciliation method in a cross-network data exchange and transmission scenario is characterized in that: The traceability mechanism triggered by verification failure is as follows: The post-data reconciliation system checks the transmission sequence number involved when the verification fails. Search for the corresponding transaction record in the alliance chain. The search function is , the return value is a transaction set containing the transmission sequence number , expressed as: ； For transaction collection Every transaction in , extract the corresponding pre-check code and timestamp, compare the corresponding pre-check code and timestamp with the post-check code generated by the post-data reconciliation system to determine whether the business data has been tampered with or lost during the transmission process.

6. A data comparison and reconciliation system in a cross-network data exchange and transmission scenario, characterized in that: The system includes a front-end data reconciliation system deployed in a front-end network and a rear-end data reconciliation system deployed in a rear-end network, and data transmission is performed through a one-way transmission channel constructed by a data exchange system, wherein: The front-end data reconciliation system is connected to the front-end business system, receives business data through the business interface, and generates verification information including a front-end verification code, a front-end business data format, a front-end business data volume and a unique identification code based on the business data, and after attaching a transmission sequence number to the verification information, re-encapsulates the business data to obtain a re-encapsulated data packet, and sends it to the rear-end data reconciliation system through the unidirectional transmission channel; The post-data reconciliation system receives the secondary encapsulated data packet and decapsulates it, and generates corresponding verification information based on the business data obtained by decapsulation, including a post-check code, a post-business data format, and a post-business data volume; obtains a comparison result by comparing the corresponding verification information with the received verification information, marks the verification status according to the comparison result, and triggers an abnormal classification and tracing mechanism if the verification fails, and performs automatic retransmission or manual intervention; Among them, generating a verification code based on business data specifically involves building a verification code generation model with the same structure in the front-end data reconciliation system and the back-end data reconciliation system. The verification code generation model extracts text, table and image features of business data based on a multimodal adversarial hash network, and dynamically selects a path to generate a verification code according to information entropy. The front-end data reconciliation system generates a front verification code, and the back-end data reconciliation system generates a back verification code, wherein: The multimodal adversarial hashing network includes a branch network, an attention fusion layer and a binarization layer. Specifically: The branch network includes a text branch layer, a table branch layer and an image branch layer. The text branch layer extracts text features through a lightweight Transformer architecture, the table branch layer outputs table features by modeling table row-column relationships through a graph attention network GAT, and the image branch layer extracts image features through a depth-separable convolution. The attention fusion layer is used to fuse text features, table features and image features, which can be expressed as: ； ； ； In the formula, For fusion features; For the The attention weights of the branches; For the The characteristics of each branch; is the text feature; is the table feature; is the image feature; is the index of the branch in the branch network; is the attention weight matrix in the attention fusion layer; The fused features are input into the binary layer of adversarial training to generate an intermediate check code, which can be expressed as: ； ； In the formula, is the intermediate check code; is the hash layer weight matrix; is the bias term of the binarization layer; Dynamically selecting a path based on information entropy is specifically to dynamically select a compression or block path in a multi-branch path tree based on the information entropy of the intermediate check code to generate the final check code, which is expressed as: ； In the formula, is the final check code. If the final check code is the pre-check code, it is recorded as , if the final check code is a post-check code, it is recorded as ; is the Sigmoid function; is the output layer attention weight; To perform splicing operations; Output for XOR compression path; Output for the block hash path; Select the corresponding bias item for the path; in: The dynamic path selection rule of the multi-branch path tree is specifically: When the information entropy of the intermediate check code is less than the preset entropy threshold, the XOR compression path is selected. The XOR compression path is expressed as: ； In the formula, is the length of the intermediate check code; To perform a bitwise XOR operation; For the Bit middle check code; For the Bit middle check code; is the position index of the intermediate check code; When the information entropy of the intermediate check code is greater than or equal to the preset entropy threshold, the block hash path is selected. The block hash path is expressed as: ； In the formula, is the number of blocks; It is a lightweight convolution operation; For the The intermediate checksum of each block; The block index.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for comparing and reconciling data in a cross-network data exchange and transmission scenario is implemented as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a data comparison and reconciliation method in a cross-network data exchange and transmission scenario is implemented as described in any one of claims 1 to 5.

Images