Tag data transmission method and device, tag data consistency verification method and device
By combining compressed sensing and digital signatures with public key verification, the problem of tag data transmission and verification resource consumption in high-concurrency, low-latency scenarios is solved, achieving efficient and secure data transmission and verification, which is suitable for Internet data centers and content delivery networks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CETC BIGDATA RES INST CO LTD
- Filing Date
- 2026-04-27
- Publication Date
- 2026-05-29
AI Technical Summary
In high-concurrency, low-latency scenarios, traditional tag data transmission and verification methods consume a large amount of computing resources, leading to competition for computing resources and low processing efficiency, which affects system reliability and user experience.
The tag data is compressed using compression sensing technology, a hash value is generated and digitally signed, and a unified verification mechanism of "compression-hash-signature" is constructed by combining public key verification and hash value comparison, and the intensive operation is offloaded by DPU hardware acceleration.
It significantly reduces transmission and computation overhead, improves the reliability and security of data transmission and verification, and is suitable for high-concurrency, low-latency Internet data scenarios.
Smart Images
Figure CN122120026A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data verification, specifically to a tag data transmission method and apparatus, and a tag data consistency verification method and apparatus. Background Technology
[0002] With the rapid development of internet services, massive amounts of user data are continuously generated and transferred in distributed storage systems, content delivery networks (CDNs), and large-scale online applications. During transmission, replication, and persistence, data is highly susceptible to damage or loss due to network jitter, hardware failures, or software anomalies, thus affecting system reliability and user experience. To ensure data integrity and reliability, data consistency verification has become a critical aspect of system operation and maintenance. Traditional verification methods often rely on general-purpose CPUs for hash calculations and comparisons.
[0003] As data volume grows exponentially, traditional verification methods consume a large amount of transmission and computing resources, and have shown bottlenecks such as competition for computing resources and low processing efficiency in high-concurrency, low-latency scenarios. Summary of the Invention
[0004] This invention provides a tag data transmission method and apparatus, and a tag data consistency verification method and apparatus, which reduce the overhead of tag data transmission and the computational overhead of data verification, and provide reliable protection for the security of tag data in high-concurrency, low-latency scenarios.
[0005] Therefore, the present invention provides the following technical solution: On one hand, the present invention provides a tag data transmission method, the method comprising: The tag data to be transmitted is compressed to obtain compressed tag data; Generate a hash value for the compressed tag data; The hash value is digitally signed using a local private key to obtain signature information; The compressed tag data and the signature information are encapsulated into a structured data packet; Send the structured data packet.
[0006] Optionally, compressing the tag data to be transmitted to obtain compressed tag data includes: using a measurement matrix that satisfies the constrained isometry condition to compress the tag data to be transmitted through linear projection to obtain compressed tag data.
[0007] Optionally, generating a unique hash value for the compressed tag data includes: generating a unique hash value for the compressed tag data using a collision-resistant and safe hash function.
[0008] Optionally, sending the structured data packet includes sending the structured data packet via a standard network protocol.
[0009] Optionally, the method further includes: sending the tag data to be transmitted to the DPU so that the DPU generates the structured data packet.
[0010] On the other hand, the present invention also provides a method for verifying the consistency of tag data, the method comprising: Obtain compressed tag data and signature information from the received structured data packets; The validity of the signature information is verified using the sender's public key; After verification, the hash value of the corresponding compressed tag data is obtained from the tag information, and the hash value of the compressed tag data is calculated using a set algorithm; If the calculated hash value matches the hash value obtained from the tag information, the compressed tag data is reconstructed to obtain tag data.
[0011] Optionally, reconstructing the compressed tag data to obtain tag data includes: reconstructing the compressed tag data using a sparse reconstruction algorithm to obtain tag data.
[0012] Optionally, the sparse reconstruction algorithm includes any one of the following: Orthogonal Matching Pursuit (OMP), Basis Pursuit (BP), and LASSO optimization.
[0013] Optionally, the method further includes sending the compressed tag data and signature information to the DPU so that the DPU generates the tag data.
[0014] On the other hand, the present invention also provides a tag data transmission device, the device comprising: The compression module is used to compress the tag data to be transmitted, resulting in compressed tag data; The hash generation module is used to generate hash values for the compressed tag data; The signature module is used to digitally sign the hash value using a local private key to obtain signature information; An encapsulation module is used to encapsulate the compressed tag data and the signature information into a structured data packet; A sending module is used to send the structured data packets.
[0015] On the other hand, the present invention also provides a tag data consistency verification device, the device comprising: The information acquisition module is used to acquire compressed tag data and signature information from the received structured data packets; The verification module is used to verify the validity of the signature information using the sender's public key; The comparison module is used to obtain the hash value corresponding to the compressed tag data from the tag information after the verification module verifies the signature information, calculate the hash value of the compressed tag data using a set algorithm, and compare whether the calculated hash value is consistent with the hash value obtained from the tag information. The tag reconstruction module is used to reconstruct the compressed tag data to obtain tag data when the comparison module determines that the calculated hash value is consistent with the hash value obtained from the tag information.
[0016] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when run by a processor, performs the steps of the tag data transmission method or the steps of the tag data consistency verification method.
[0017] The tag data transmission method and apparatus provided by this invention achieve efficient compression of tag data through compressed sensing technology, which can significantly reduce transmission overhead while ensuring information integrity, and can be better applied to high-concurrency, low-latency Internet data scenarios.
[0018] The tag data consistency verification method and apparatus provided by the present invention, by constructing an integrated verification mechanism of "compression-hash-signature", combined with public key verification and hash value comparison, can take into account the authenticity, integrity and anti-tampering ability of data, and improve the reliability of security verification.
[0019] Furthermore, by introducing a DPU to offload computation, hardware acceleration of intensive operations such as compression, hash value generation, and signing can significantly reduce CPU load and processing latency, achieving high performance and scalability for data consistency verification. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.
[0021] Figure 1 This is a flowchart of a tag data transmission method provided in an embodiment of the present invention; Figure 2 This is a flowchart of a tag data consistency verification method provided in an embodiment of the present invention; Figure 3This is a schematic diagram of a tag data transmission device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a tag data consistency verification device provided in an embodiment of the present invention. Detailed Implementation
[0022] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.
[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0024] To address the issue of high resource consumption in data transmission and consistency verification under high concurrency and low latency scenarios, this invention provides a tag data transmission method and apparatus, as well as a tag data consistency verification method. These methods significantly reduce transmission overhead and improve the reliability of security verification while ensuring information integrity, thus better meeting the needs of high-concurrency and low-latency Internet applications.
[0025] like Figure 1 The diagram shown is a flowchart of a tag data transmission method provided in an embodiment of the present invention, which includes the following steps: Step 101: Compress the tag data to be transmitted to obtain compressed tag data.
[0026] Tag data typically exhibits high-dimensional sparsity or compressibility. It can be represented as: ,in, Represents the set of real numbers. n It is a positive integer representing the dimension. Indicates that it consists of all contained n An ordered array of n real numbers (called n A set consisting of tuples.
[0027] To reduce transmission bandwidth and storage overhead, compressed sensing (CS) theory can be introduced to compress the tag data.
[0028] The core idea of compressed sensing is that if a signal is sparse (i.e., compressible) in a certain transform domain, then the signal can be acquired through non-adaptive linear measurements at a sampling rate much lower than that required by the Nyquist sampling theorem, and it can be reconstructed accurately with a high probability.
[0029] Therefore, in a non-limiting embodiment, a measurement matrix that satisfies the Restricted Isometry Property (RIP) condition can be selected. ,in Data compression is achieved through linear projection. The compressed label data is denoted as: .
[0030] The RIP condition ensures that the measurement matrix maintains the structure of the sparse signal; that is, during dimensionality reduction (from N dimensions to M dimensions), two different sparse signals cannot be mapped to the same set of measurements. The RIP condition guarantees the stability and reversibility of the compression operation, laying a theoretical foundation for subsequent high-fidelity reconstruction.
[0031] In practice, random matrices (such as Gaussian random matrices, Bernoulli matrices, etc.) can be used to compress the tag data to be transmitted, resulting in compressed tag data. This process can map high-dimensional data to a low-dimensional observation space while preserving the key structural information of the original tag data. .
[0032] By compressing tag data, the amount of data transmitted can be significantly reduced, communication efficiency can be improved, and the semantic integrity of the tag data can be preserved.
[0033] Step 102: Generate a hash value for the compressed tag data.
[0034] In one non-limiting embodiment, a collision-resistant and secure hash function H(·) (such as SHA-256) can be used to generate a unique hash value for the compressed tag data y: h=H(y). This hash value h serves as a data fingerprint and is highly sensitive to any minor tampering.
[0035] In some embodiments, other methods can also be used to generate hash values corresponding to the compressed tag data, such as statistical methods, which count the frequency of occurrence of all tags and select the K tags with the highest frequency to generate hash values; or model-based methods, etc., which are not limited in this embodiment of the invention.
[0036] Step 103: Use the local private key to digitally sign the hash value to obtain the signature information.
[0037] Specifically, the sender can use the local private key SK to digitally sign the generated hash value (e.g., the hash value) to obtain the signature information sign, denoted as: sign=Sign(SK, h).
[0038] This signature process can be based on asymmetric encryption systems, such as the RSA (Rivest–Shamir–Adleman) encryption algorithm or ECDSA (Elliptic Curve Digital Signature Algorithm), to achieve identity binding and non-repudiation, ensuring the integrity and authenticity of the compressed data. Even if an attacker intercepts the data packet, they cannot forge a valid signature, thus ensuring the authenticity and integrity of the data during transmission from a cryptographic perspective.
[0039] Step 104: Encapsulate the compressed tag data and signature information into a structured data packet.
[0040] After compression and signing are completed, the key information can be encapsulated into a structured data packet {y, sign}, where y is the compressed data and {sign} is the digital signature.
[0041] The encapsulation mechanism is designed following the principle of minimum information exposure, ensuring full functionality while reducing the risk of privacy leaks. The structured data packets encapsulated with key information balance security and parsability, avoiding the plaintext exposure of the original tag data while providing the receiver with all the parameters required for verification.
[0042] Step 105: Send structured data packets.
[0043] Specifically, the structured data packets can be transmitted via standard network protocols such as TCP / IP or UDP. This is suitable for various internet architectures, including intra-data center communication, edge-cloud collaboration, and content delivery networks.
[0044] Of course, in some specific applications, non-standard transmission protocols agreed upon by the sender and receiver may also be used, and this embodiment of the invention does not limit this.
[0045] In some embodiments, it is also possible to Figure 1 In the illustrated embodiment, some key computational tasks are offloaded to the DPU (Data Processing Unit) for hardware acceleration. Leveraging the DPU's dedicated hardware acceleration capabilities and low-overhead parallel processing architecture, the CPU's computational resource consumption is reduced, and processing speed is improved. For example, the tag data to be transmitted can be sent to the DPU so that the DPU generates the structured data packet; or some functions in steps 101 to 104 can be offloaded to the DPU to complete the task.
[0046] The tag data transmission method provided in this invention achieves efficient compression of tag data through compressed sensing technology, which can significantly reduce transmission overhead while ensuring information integrity, and is better suited for high-concurrency, low-latency Internet data scenarios.
[0047] In response to the aforementioned compressed tag data, this embodiment of the invention also provides a tag data consistency verification method, which enables effective verification of tag data upon receipt of the structured data packet by the receiver.
[0048] like Figure 2 The diagram shown is a flowchart of a tag data consistency verification method provided in an embodiment of the present invention, which includes the following steps: Step 201: Obtain compressed tag data and signature information from the received structured data packet.
[0049] The structured data packet encapsulates compressed tag data and signature information. The structured data packet can be decapsulated using a decapsulation method corresponding to the encapsulation method to obtain the compressed tag data and signature information.
[0050] Step 202: Verify the validity of the signature information using the sender's public key.
[0051] Specifically, the validity of the signature can be determined using a pre-defined verification algorithm, resulting in a Valid verification result. Valid = Verify(PK, sign); Here, PK represents the sender's public key.
[0052] If the verification result Valid is true, it indicates that the signature was generated by the legitimate sender holding the corresponding private key, and the signed content was not modified during transmission.
[0053] This step effectively defends against man-in-the-middle attacks, identity forgery, and replay attacks, forming the first line of defense for security verification.
[0054] It should be noted that the verification process must strictly rely on a Public Key Infrastructure (PKI) or a trusted identity management system to ensure the authority and robustness of the authentication mechanism. The verification process can be performed by the recipient or a trusted third-party organization; this embodiment of the invention does not limit this.
[0055] Step 203: After verification, obtain the hash value of the corresponding compressed tag data from the tag information, and calculate the hash value of the compressed tag data using the set algorithm.
[0056] After the signature verification is successful, the receiver independently calculates the hash value of the received compressed tag data y (e.g., hash value h'=H(y)) and compares it with the original hash value (e.g., hash value h) decrypted from the signature information. If they match, it further confirms that the data has not undergone bit-level tampering or transmission errors during the transmission link.
[0057] By employing a dual verification mechanism—namely, verifying the signature first and then the hash value—a defense-in-depth structure can be formed, which can prevent malicious attacks and detect accidental data corruption, significantly improving the overall reliability of the data.
[0058] The hash uses hash values, which makes the comparison operation more lightweight and efficient, suitable for high-frequency batch processing, and can better meet the real-time requirements of Internet scenarios.
[0059] Step 204: If the calculated hash value is consistent with the hash value obtained from the tag information, then the compressed tag data is reconstructed to obtain the tag data.
[0060] Reconstructing compressed tag data refers to decompressing and restoring the tag data to obtain the original tag data.
[0061] Specifically, the receiver can use a sparse reconstruction algorithm to reconstruct the compressed tag data to obtain the tag data. The sparse reconstruction algorithm may include, but is not limited to, any of the following: Orthogonal Matching Pursuit (OMP), Basis Pursuit (BP), Least Absolute Shrinkage and Selection Operator (LASSO) optimization algorithm, etc.
[0062] This reconstruction process utilizes sparse priors of the original data to achieve high-precision signal recovery while satisfying RIP (Record Integrity Index). Reconstruction quality depends on the compression ratio m / n, measurement matrix design, and noise level, and can typically be quantified using Peak Signal-to-Noise Ratio (PSNR) or Structural Similarity Index (SSIM). The final output D' approximates the original labeled data D.
[0063] Similarly, it can also be Figure 2 In the illustrated embodiment, some critical computational tasks are offloaded to DPU hardware acceleration. Leveraging the DPU's dedicated hardware acceleration capabilities and low-overhead parallel processing architecture, the consumption of CPU computing resources is reduced, thus improving processing speed. For example, the above... Figure 2 All or some of the steps in the process are unloaded to the DPU to complete.
[0064] With its parallel architecture, dedicated instruction set, and low-latency memory access capabilities, the DPU can significantly accelerate matrix operations and cryptographic operations. Simultaneously, through zero-copy shared memory and event-driven interrupt mechanisms, it achieves efficient collaboration between the DPU and the host CPU, reducing context switching and data copying overhead. This offloading strategy not only frees up main CPU resources to handle core business logic but also reduces overall verification latency by an order of magnitude, better supporting large-scale, low-latency internet data consistency assurance systems.
[0065] The tag data consistency verification method provided in this invention constructs an integrated "compression-hash-signature" verification mechanism, combining public key verification and hash value comparison, which can take into account data authenticity, integrity and anti-tampering ability, and improve the reliability of security verification.
[0066] Accordingly, embodiments of the present invention also provide a tag data transmission device, such as... Figure 3 The diagram shown is a structural schematic of a tag data transmission device provided in an embodiment of the present invention.
[0067] The tag data transmission device 300 includes the following modules: Compression module 301 is used to compress the tag data to be transmitted to obtain compressed tag data; Hash generation module 302 is used to generate hash values for the compressed tag data; The signature module 303 is used to digitally sign the hash value using a local private key to obtain signature information; Encapsulation module 304 is used to encapsulate the compressed tag data and the signature information into a structured data packet; The sending module 305 is used to send the structured data packet.
[0068] Accordingly, embodiments of the present invention also provide a tag data consistency verification device, such as... Figure 4 The diagram shown is a structural schematic of a tag data consistency verification device provided in an embodiment of the present invention.
[0069] The tag data consistency verification device 400 includes the following modules: The information acquisition module 401 is used to acquire compressed tag data and signature information from the received structured data packet; Verification module 402 is used to verify the validity of the signature information using the sender's public key; The comparison module 403 is used to obtain the hash value corresponding to the compressed tag data from the tag information after the verification module verifies the signature information, calculate the hash value of the compressed tag data using a set algorithm, and compare whether the calculated hash value is consistent with the hash value obtained from the tag information. The tag reconstruction module 404 is used to reconstruct the compressed tag data to obtain tag data when the comparison module determines that the calculated hash value is consistent with the hash value obtained from the tag information.
[0070] The specific implementation methods of the above modules can be referred to the descriptions in the corresponding method embodiments above, and will not be repeated here.
[0071] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.
[0072] The present invention also provides a storage medium, which is a computer-readable storage medium storing a computer program thereon, the computer program being executable when it runs. Figure 1 or Figure 2 The method shown may include some or all of the steps. The storage medium may include read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc. The storage medium may also include non-volatile memory or non-transitory memory, etc.
[0073] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data provider to another website, computer, server, or data provider via wired or wireless means.
[0074] The embodiments of the present invention have been described in detail above. Specific implementation methods have been used to illustrate the present invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and apparatus of the present invention, and are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention, and the content of this specification should not be construed as a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A tag data transmission method, characterized in that, The method includes: The tag data to be transmitted is compressed to obtain compressed tag data; Generate a hash value for the compressed tag data; The hash value is digitally signed using a local private key to obtain signature information; The compressed tag data and the signature information are encapsulated into a structured data packet; Send the structured data packet.
2. The tag data transmission method according to claim 1, characterized in that, The compression of the tag data to be transmitted to obtain compressed tag data includes: By using a measurement matrix that satisfies the constrained isometry condition, the tag data to be transmitted is compressed through linear projection to obtain compressed tag data.
3. The tag data transmission method according to claim 1, characterized in that, The process of generating a unique hash value for the compressed tag data includes: A collision-resistant and safe hash function is used to generate a unique hash value for the compressed tag data.
4. The tag data transmission method according to claim 1, characterized in that, The structured data packet is sent; The structured data packets are sent using standard network protocols.
5. The tag data transmission method according to any one of claims 1 to 4, characterized in that, The method further includes: The tag data to be transmitted is sent to the DPU so that the DPU generates the structured data packet.
6. A method for verifying the consistency of tag data, characterized in that, The method includes: Obtain compressed tag data and signature information from the received structured data packets; The validity of the signature information is verified using the sender's public key; After verification, the hash value of the corresponding compressed tag data is obtained from the tag information, and the hash value of the compressed tag data is calculated using a set algorithm; If the calculated hash value matches the hash value obtained from the tag information, the compressed tag data is reconstructed to obtain tag data.
7. The tag data consistency verification method according to claim 6, characterized in that, The process of reconstructing the compressed tag data to obtain tag data includes: The compressed tag data is reconstructed using a sparse reconstruction algorithm to obtain the tag data.
8. The tag data consistency verification method according to claim 7, characterized in that, The sparse reconstruction algorithm includes any one of the following: Orthogonal Matching Pursuit (OMP), Basis Pursuit (BP), and LASSO optimization.
9. The tag data consistency verification method according to any one of claims 6 to 8, characterized in that, The method further includes: The compressed tag data and signature information are sent to the DPU so that the DPU generates the tag data.
10. A tag data transmission device, characterized in that, The device includes: The compression module is used to compress the tag data to be transmitted, resulting in compressed tag data; The hash generation module is used to generate hash values for the compressed tag data; The signature module is used to digitally sign the hash value using a local private key to obtain signature information; An encapsulation module is used to encapsulate the compressed tag data and the signature information into a structured data packet; A sending module is used to send the structured data packets.
11. A tag data consistency verification device, characterized in that, The device includes: The information acquisition module is used to acquire compressed tag data and signature information from the received structured data packets; The verification module is used to verify the validity of the signature information using the sender's public key; The comparison module is used to obtain the hash value corresponding to the compressed tag data from the tag information after the verification module verifies the signature information, and to calculate the hash value of the compressed tag data using a set algorithm, and compare whether the calculated hash value is consistent with the hash value obtained from the tag information; The tag reconstruction module is used to reconstruct the compressed tag data to obtain tag data when the comparison module determines that the calculated hash value is consistent with the hash value obtained from the tag information.
12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it executes the steps of the tag data transmission method according to any one of claims 1 to 5, or the steps of the tag data consistency verification method according to any one of claims 6 to 9.