Data processing method, apparatus and system

Abstract

The application provides a data processing method, device and system, and applies to the technical field of communication. In the method, an N data blocks can be acquired by a generation party, and a first data set can be generated according to the N data blocks. In addition to the N data blocks, the first data set can further include a first mapping sequence, and the verification information of each data block in the N data blocks corresponds to at least one specific element in the first mapping sequence, and the verification information of each data block is generated according to each data block and a corresponding counting sequence. Based on the scheme, if the audit granularity is set to be consistent with the data block size, fine-grained audit can be realized. Moreover, the verification information of each data block is generated in combination with the counting sequence, so that the verification information is difficult to fake, data fraud can be effectively prevented, and the reliability of the audit result can be improved. In addition, the verification information is mapped to the first mapping sequence, and the first mapping sequence is used for verification, so that the storage and communication resources can be saved.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a data processing method, apparatus and system. Background Technology

[0002] In data transmission scenarios, the parties involved in data transmission can audit the transmitted content to check and verify its authenticity, integrity, and reliability.

[0003] With increasing demands for the authenticity, integrity, and reliability of transmitted content, more and more transmission scenarios require fine-grained auditing. However, the reliability of current methods for achieving fine-grained auditing needs improvement. Summary of the Invention

[0004] This application provides a data processing method, apparatus, and system to address the problem of poor accuracy and security in existing methods for implementing fine-grained auditing.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, a data processing method is provided, which can be applied to a data generator. This method includes: first, the generator acquires N data blocks, where N is a positive integer. Then, the generator determines a first dataset, which may include the N data blocks and a first mapping sequence. The verification information of the nth data block corresponds to at least one specific element in the first mapping sequence. The first mapping sequence can be used to verify whether the nth data block has been modified. The verification information of the nth data block is determined based on the nth data block and its corresponding counting sequence. Different data blocks correspond to different counting sequences, where n is a positive integer from 1 to N.

[0007] Based on this scheme, the generator can divide the data into N data blocks and generate verification information for each data block, mapping it to a first mapping sequence. This allows the first dataset generated by the generator to support fine-grained auditing. Furthermore, mapping the verification information to the mapping sequence eliminates the need to store N separate verification messages, saving storage resources. Additionally, the verification information generated by the generator for each data block is combined with a corresponding counting sequence for each data block, and different data blocks correspond to different counting sequences. This ensures that the verification information generated by the generator for each data block is unique and difficult to forge, improving information security and effectively preventing illegal attacks. Because the verification information is difficult to forge, the reliability of the verification results obtained when the verifier verifies the dataset is also greatly improved.

[0008] In conjunction with the first aspect described above, in one possible implementation, the data processing method may further include: the generator determining a digest of the counting sequence corresponding to the nth data block based on the first key. Then, the generator performs calculations on the digest of the counting sequence corresponding to the nth data block and the nth data block to obtain the verification information of the nth data block.

[0009] Based on this scheme, when generating verification information for a data block by combining the counting sequence, the generator can use a key to calculate the digest of the counting sequence, and then calculate the verification information for the data block based on the digest. This method uses a key to calculate the digest of the counting sequence, which increases the security of the verification information and makes it more difficult for attackers to forge information.

[0010] In conjunction with the first aspect described above, in one possible implementation, the data processing method may further include: first, mapping the verification information of the nth data block to a Bloom filter using a set of hash functions to obtain an intermediate mapping sequence; then, determining a first mapping sequence based on the intermediate mapping sequence. The set of hash functions includes at least one hash function.

[0011] Based on this scheme, the first mapping sequence in this application can be implemented by a Bloom filter. A Bloom filter can be used to verify whether an input is in a certain set. Therefore, the first mapping sequence can be used to verify whether an input of the Bloom filter belongs to the verification information of the nth data block.

[0012] In conjunction with the first aspect mentioned above, in one possible implementation, at least one hash function is a keyed hash function. Based on this scheme, the security of the mapping process can be improved, preventing the mapping rules from being reverse-engineered.

[0013] In conjunction with the first aspect mentioned above, one possible implementation involves determining the first mapping sequence based on the intermediate mapping sequence. Specifically, this can include: encrypting the intermediate mapping sequence using the second key to obtain the first mapping sequence. Alternatively, the intermediate mapping sequence can be determined as the first mapping sequence. Based on this scheme, whether or not to encrypt the intermediate mapping sequence can be chosen according to different security requirements.

[0014] In conjunction with the first aspect described above, in one possible implementation, the data processing method may further include: determining a summary of the first mapping sequence, wherein the first dataset also includes a summary of the first mapping sequence. It should be understood that this approach is intended to protect the integrity of the first mapping sequence, and the summary of the first mapping sequence can be used to verify whether the first mapping sequence has been modified.

[0015] In conjunction with the first aspect described above, in one possible implementation, the first dataset also includes a counting sequence corresponding to the first data block among the N data blocks. Alternatively, the first dataset also includes a counting sequence corresponding to each of the N data blocks. Based on this scheme, the counting sequences can be carried in the first dataset, enabling the verifier to determine the counting sequence corresponding to each data block during verification, thereby correctly verifying the corresponding data blocks.

[0016] In conjunction with the first aspect described above, in one possible implementation, the data processing method may further include: sending a first dataset. In other words, the method of this application can be applied to data transmission scenarios, where the data generator can be the data sender in the data transmission scenario. Based on the data processing method provided by this application, the security and unforgeability of the transmitted data can be improved, effectively preventing data deception. Furthermore, since the verification information is mapped to the first mapping sequence, it is not necessary to carry verification information for each data block, thereby saving communication resources.

[0017] Secondly, a data processing method is provided that can be applied to a data updater. This method may include: first, the updater obtains a second dataset; then, the updater updates the second dataset to obtain an updated second dataset, wherein the updated second dataset and the updated second dataset have at least one different data block; subsequently, the updater may determine a third dataset, which includes the updated second dataset and an operation record block used to instruct the updater to update the second dataset.

[0018] Based on this scheme, the updater can update the data blocks in the second dataset, and after updating the dataset, it can generate an operation record block indicating the update, so that the verifier can know the updater's update during the subsequent verification.

[0019] In conjunction with the second aspect above, in one possible implementation, updating the second dataset may specifically include: adding a first data block to the second dataset, the operation record block including first location information, the first location information being used to indicate the position of the first data block in the third dataset, and updating the position relative to the second dataset.

[0020] This scheme is an update method that can be executed by the updater. For newly added data blocks, the updater can record their location information, so that even if the updater can update the dataset, the data blocks in the dataset are traceable.

[0021] In conjunction with the second aspect above, in one possible implementation, the operation record block further includes a counting sequence corresponding to the first data block and a second mapping sequence. The counting sequence corresponding to the first data block is used to determine the verification information of the first data block, and the verification information of the first data block corresponds to at least one specific element in the second mapping sequence.

[0022] Based on this scheme, a new counting sequence can be assigned to newly added data blocks, as well as verification information and mapping sequences can be generated for them, which can be used for subsequent verification of the newly added data blocks.

[0023] In conjunction with the second aspect described above, in one possible implementation, updating the second dataset includes: modifying a second data block within the second dataset. Modifying the second data block includes at least one of the following: replacing part or all of the content of the second data block, adding content to the second data block, or deleting part of the content of the second data block. The operation record block includes second location information, which indicates the position of the second data block within the third dataset and updates its position relative to the second dataset.

[0024] This scheme is another update method that the updater can execute. The updater can record the location information of the modified data blocks, so that even if the updater can update the dataset, the data blocks in the dataset are traceable.

[0025] In conjunction with the second aspect above, in one possible implementation, the operation record block further includes a counting sequence corresponding to the second data block and a third mapping sequence. The counting sequence corresponding to the second data block is used to determine the verification information of the second data block, and the verification information of the second data block corresponds to at least one specific element in the third mapping sequence.

[0026] Based on this scheme, a new counting sequence can be assigned to the modified data block, and new verification information and mapping sequences can be regenerated for it, which can be used for subsequent verification of the modified data block.

[0027] In conjunction with the second aspect above, in one possible implementation, updating the second dataset includes: deleting a third data block from the second dataset, wherein the operation record block includes third location information, which is used to indicate the position of the update relative to the second dataset.

[0028] This scheme is another update method that the updater can execute. The updater can record the location information of deleted data blocks, so that even if the updater can update the dataset, the data blocks in the dataset are traceable.

[0029] In conjunction with the second aspect above, in one possible implementation, the operation record block further includes at least one of the following: the identifier of the updater, the time when the update occurred or ended, the type of update, the source of the second dataset, or the signature of the operation record block; wherein the source of the second dataset is the device that generated the second dataset.

[0030] In conjunction with the second aspect described above, in one possible implementation, after determining the third dataset, the method further includes sending the third dataset. In other words, this method can be applied to data transmission scenarios, where the updater can be middleware in the data transmission process.

[0031] Thirdly, a data processing method is provided that can be applied to a data verifier. This method includes: First, the verifier obtains a fourth dataset, which comprises M data blocks and a first mapping sequence, where M is a positive integer. Then, the verifier determines K data blocks from the M data blocks, where K is a positive integer less than or equal to M. Next, the verifier determines the verification information of the k-th data block based on the k-th data block and its corresponding counting sequence, where k is a positive integer from 1 to K. Additionally, the verifier can determine a fourth mapping sequence, where the verification information of the k-th data block corresponds to at least one specific element in the fourth mapping sequence. Finally, the verifier determines whether the k-th data block has been modified based on the first and fourth mapping sequences.

[0032] Based on this scheme, the verifier can identify K data blocks in the acquired dataset and generate a fourth mapping sequence mapping the verification information of these K operation record blocks in the same way as the generator. Then, based on the first and fourth mapping sequences, it can determine whether the K data blocks without operation records have truly not been modified. This method processes data at the data block level, making it suitable for fine-grained auditing. Furthermore, since the verification information is generated by combining the counting sequence of data blocks, data deception can be effectively prevented, significantly reducing the error rate of the verifier's audit and greatly improving the reliability of the verification results.

[0033] In conjunction with the third aspect mentioned above, in one possible implementation, the fourth dataset further includes J operation record blocks, each corresponding one-to-one with one of the J updaters. The operation record blocks are used to instruct the updater that generated the operation record block to update the original dataset, which is the dataset before the updater made the update. The verifier determines K data blocks out of the M data blocks, specifically including: determining K data blocks out of the M data blocks based on the J operation record blocks.

[0034] Based on this scheme, the verifier can determine K data blocks based on the operation record blocks, thereby enabling the verifier to identify M data blocks with modified records and those without modified records, achieving refined auditing of the dataset.

[0035] In conjunction with the third aspect mentioned above, in one possible implementation, determining K data blocks from M data blocks based on J operation record blocks can specifically include: determining the target data block corresponding to the update data block of the j-th updater among the M data blocks based on the j-th operation record block. Here, the update data block is a data block in the second original dataset indicated by the j-th operation record block, and the second original dataset is obtained after the j-th updater updates the first original dataset; the initial value of j is J. If j-1 is not equal to 0, determining the position of the data blocks in the first original dataset excluding the target data block corresponding to the update data block among the M data blocks based on the j-th operation record block, and decrementing j by 1. If j-1 is equal to 0, determining K data blocks from the M data blocks excluding the target data block corresponding to the update data block.

[0036] Based on this scheme, the updated data blocks of each updater can be determined by working backward from the last operation record block, and thus the K data blocks out of the M data blocks that have not been operated on can be identified. Therefore, the verifier verifies the K data blocks without operation records. It can be seen that the method provided in this application allows the verifier to further verify the data blocks without operation records to determine whether these data blocks have been illegally tampered with.

[0037] In conjunction with the third aspect mentioned above, in one possible implementation, the updated data block is the data block added or modified by the j-th updater. The j-th operation record block includes fourth position information, which indicates the position of the updated data block in the second original dataset and the position of the update performed by the updater relative to the first original dataset.

[0038] In conjunction with the third aspect described above, in one possible implementation, the j-th operation record block includes a counting sequence corresponding to the updated data block and a fifth mapping sequence, where the verification information of the updated data block corresponds to at least one specific element in the fifth mapping sequence. The data processing method may further include: determining the verification information of the target data block corresponding to the updated data block based on the target data block and the counting sequence corresponding to the updated data block; determining a sixth mapping sequence, where the verification information of the updated data block corresponds to at least one specific element in the sixth mapping sequence; and verifying whether the target data block has been modified after the update is performed by the j-th updater based on the fifth mapping sequence and the sixth mapping sequence.

[0039] Based on this scheme, the verifier can further verify data blocks with operation records to check whether there are any unrecorded update behaviors. For example, it can determine whether the data block has been illegally tampered with after being updated by the party being updated.

[0040] In conjunction with the third aspect described above, in one possible implementation, the fourth dataset may further include an initial counting sequence. After determining K data blocks out of the M data blocks, the data processing method may further include: determining the counting sequence corresponding to each of the K data blocks based on the initial counting sequence. It should be understood that determining the counting sequence corresponding to the data block is necessary for correct verification when validating the data blocks.

[0041] Fourthly, a data processing apparatus is provided for implementing the above-described method. This data processing apparatus can be the generator of the first aspect, the updater of the second aspect, or the verifier of the third aspect. The data processing apparatus may include modules, units, or means corresponding to the above-described method. These modules, units, or means can be implemented in hardware, software, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above-described functions.

[0042] In one possible implementation, the data processing device includes a processing module and a transceiver module. The transceiver module is used to perform message receiving and sending operations performed by the data processing device in the methods described in the first, second, or third aspects above. The processing module is used to invoke instructions to perform message processing or control operations performed by the data processing device in the methods described in the first, second, or third aspects above.

[0043] Fifthly, a data processing apparatus is provided, comprising: a processor; the processor being coupled to a memory and, after reading computer instructions stored in the memory, executing the method described in the first, second, or third aspect above according to the instructions.

[0044] In one possible implementation, the data processing device further includes a memory for storing computer instructions.

[0045] In one possible implementation, the data processing device further includes a communication interface for communicating with other devices. For example, the communication interface may be a transceiver, an input / output interface, an interface circuit, an output circuit, an input circuit, a pin, or related circuitry.

[0046] In one possible implementation, the data processing device can be a chip or a chip system. When the data processing device is a chip system, it can be composed of chips or may include chips and other discrete components.

[0047] In one possible implementation, when the data processing device is a chip or chip system, the aforementioned communication interface can be an input / output interface, interface circuit, output circuit, input circuit, pin, or related circuit on the chip or chip system. The aforementioned processor can also be embodied as a processing circuit or logic circuit.

[0048] A sixth aspect provides a computer-readable storage medium storing instructions that, when executed on a computer, enable the computer to perform the methods described in the first, second, or third aspects above.

[0049] In a seventh aspect, a computer program product containing instructions is provided, which, when run on a computer, enables the computer to perform the methods described in the first, second, or third aspects above.

[0050] Eighthly, a communication system is provided, comprising a generator that performs the data processing method described in the first aspect, an updater that performs the data processing method described in the second aspect, and an updater that performs the data processing method described in the third aspect.

[0051] The technical effects of any possible implementation of aspects four through eight can be found in the technical effects of different implementations of aspects one, two, or three above, and will not be repeated here. Attached Figure Description

[0052] Figure 1 A schematic diagram illustrating an end-to-end transmission scenario provided in an embodiment of this application;

[0053] Figure 2 This is a schematic diagram illustrating the application of a message verification code (MAC) in an embodiment of this application.

[0054] Figure 3 A schematic diagram illustrating a transmission content audit based on application-layer semantic analysis, provided as an embodiment of this application;

[0055] Figure 4 A schematic diagram illustrating key distribution based on McTLS, provided as an embodiment of this application;

[0056] Figure 5 This is a schematic diagram illustrating end-to-end transmission based on MaTLS, provided as an embodiment of this application.

[0057] Figure 6 This application provides a schematic diagram of the structure of a communication system according to an embodiment of the present application.

[0058] Figure 7 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0059] Figure 8 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0060] Figure 9 A schematic diagram illustrating the encryption of plaintext blocks into ciphertext blocks, provided as an embodiment of this application;

[0061] Figure 10 This is a schematic diagram illustrating the generation of verification information for a data block based on a data block, as provided in an embodiment of this application.

[0062] Figure 11 This is a schematic diagram illustrating the mapping of verification information to an intermediate mapping sequence, as provided in an embodiment of this application.

[0063] Figure 12 A schematic diagram of a process for generating a first mapping sequence is provided for an embodiment of this application;

[0064] Figure 13 This is a schematic diagram of the structure of a TLS message provided in an embodiment of this application;

[0065] Figure 14 A schematic diagram illustrating a process for generating a TLS message, provided as an embodiment of this application;

[0066] Figure 15 A flowchart illustrating another data processing method provided in this application embodiment;

[0067] Figure 16 This is a schematic diagram illustrating an updater method for updating a dataset, as provided in an embodiment of this application.

[0068] Figure 17 A schematic diagram illustrating another method for updating a dataset provided in this application embodiment;

[0069] Figure 18 A schematic diagram illustrating another method for updating a dataset provided in this application embodiment;

[0070] Figure 19 A schematic diagram illustrating another method for updating a dataset provided in this application embodiment;

[0071] Figure 20 A schematic diagram illustrating another method for updating a dataset provided in this application embodiment;

[0072] Figure 21A schematic diagram illustrating another method for updating a dataset provided in this application embodiment;

[0073] Figure 22 A schematic diagram illustrating another method for updating a dataset provided in this application embodiment;

[0074] Figure 23 This is a schematic diagram of the structure of an operation record block provided in an embodiment of this application;

[0075] Figure 24 A schematic diagram illustrating multiple updaters participating in an update, as provided in an embodiment of this application;

[0076] Figure 25 A schematic diagram illustrating a scenario where multiple middleware components participate in data transmission, provided as an embodiment of this application;

[0077] Figure 26 This is a schematic diagram illustrating hierarchical encryption of the content of an operation record block, provided as an embodiment of this application.

[0078] Figure 27 A schematic diagram of a key tree provided for an embodiment of this application;

[0079] Figure 28 A flowchart illustrating yet another data processing method provided in this application embodiment;

[0080] Figure 29 This is a schematic diagram of another data processing device provided in an embodiment of this application. Detailed Implementation

[0081] Before introducing the embodiments of this application, some related technologies involved in the embodiments of this application will be explained and described. It should be noted that the following explanations and descriptions are for the purpose of making the embodiments of this application easier to understand, and should not be regarded as a limitation on the scope of protection claimed by the embodiments of this application.

[0082] Auditing refers to the inspection and verification of the accuracy and completeness of an objective. Auditing data can detect whether it conforms to established standards, benchmarks, and auditing principles, preventing false data and deceptive practices. In network transmission scenarios, auditing of transmitted content is a key technology for ensuring the authenticity, completeness, controllability, availability, and non-repudiation (non-repudiation) of the transmitted content received by the receiving end. Participants in network transmission can inspect and verify the transmitted content, thereby achieving functions such as detecting illegal tampering, tracing historical behavior, and proving malicious operations.

[0083] A common scenario for network transmission is end-to-end transmission, where the participants include the sender and the receiver. Optionally, end-to-end transmission can also involve middleware (MB), such as... Figure 1This is an end-to-end transmission scenario involving middleware, where the middleware itself is also a participant. In this scenario, the receiving end of the transmitted content performs auditing. The auditing objectives can include: whether the data has been modified, by whom the data was modified, and at what location the data was modified. Figure 1 As shown, the transmitted content is sent from the server to the client via middleware MB1 and MB2. The client can audit the transmitted content after receiving it.

[0084] In one alternative implementation, a message authentication code (MAC) can be used to audit the transmitted content. A MAC is a short data segment generated by processing the target data using a specific algorithm. The MAC can be used to check whether the target data has been altered during transmission. If the MAC calculated by the receiving end after receiving the data matches the MAC of the sending end, it indicates that the target data has not been altered. Additionally, the MAC can also serve as authentication for the sending end, allowing the receiving end to verify the origin of the target data.

[0085] For example, Figure 2 This is a diagram illustrating content auditing using MAC addresses, such as... Figure 2 As shown, after generating the transmission content, the sending end can perform calculations on the transmission content using a key and a MAC algorithm to determine the MAC address of the transmission content. Then, it can send the transmission content and its MAC address together to the receiving end. Upon receiving the transmission content, the receiving end can generate a new MAC address for the received content using the same key and algorithm. The receiving end can then compare the new MAC address with the MAC address sent by the sending end to determine if the transmission content has been modified. It should be understood that the sending and receiving ends can pre-negotiate the key and algorithm used to generate the MAC address.

[0086] It should be noted that the algorithm used to generate the MAC can include cryptographic hash functions and block cipher algorithms. For example, the cryptographic hash function can be a hash-based message authentication code (HMAC), and the block cipher algorithm can be an algorithm that encrypts the MAC using cipher block chaining (CBC) (CBC-MAC), or a one-key CBC-MAC algorithm (OMAC), etc.

[0087] It should be noted that MAC applications can efficiently detect malicious tampering of transmitted content in end-to-end transmission scenarios, while also providing proof of the authenticity of the content sender. Specifically, the end-to-end transmission scenarios for MAC applications can be based on end-to-end secure transmission protocols such as Transport Layer Security (TLS) and Internet Protocol Security (IPSec).

[0088] However, auditing transmitted content based on MAC addresses has the following drawbacks: First, the audit granularity of this method is too coarse to meet fine-grained auditing requirements. Message authentication codes generally perform a digest operation on the entire transmitted content. The receiving end can determine that the transmitted content has been modified based on MAC inconsistencies, but it cannot detect the specific scope of modifications to the transmitted content in a fine-grained manner. Second, this method cannot provide non-repudiation, especially in scenarios involving middleware. If the middleware (such as a firewall) also knows the key and algorithm for generating the MAC address, it can regenerate the MAC address of the transmitted content while modifying it. This makes it impossible for the receiving end to determine whether the received transmitted content was sent by the sender or modified by the middleware, allowing the middleware to deny its modification.

[0089] Another alternative implementation involves auditing the transmitted content based on application-layer semantic analysis. For example, the receiving end can analyze and verify the semantic information of the received transmitted content at its firewall level to achieve auditing. In this case, application-layer semantic analysis involves reviewing the entire transmitted content.

[0090] For example, Figure 3 This is a schematic diagram of a transmission content auditing method based on application-layer semantic analysis, such as... Figure 3 As shown, a client can configure a firewall when communicating with the Internet. This firewall can perform auditing functions. When the client receives content transmitted from the Internet, the firewall can perform semantic analysis and verification on the transmitted content. Taking auditing the content of a video website as an example, application-layer semantic analysis can be used to check whether the video quality does not meet expectations, whether the webpage displays errors, and whether the webpage contains advertisements, etc.

[0091] It should be noted that while this method provides fine-grained content auditing, its application-layer semantic analysis of transmitted content is highly complex. Furthermore, in client-side firewall deployments, the client needs to cooperate with the firewall's certificate implantation and trust the firewall. This method cannot detect or prevent malicious tampering or malicious behavior by the firewall. For example, in scenarios where the firewall is outsourced to a third party, the client is unaware of the firewall's behavior. Moreover, as security requirements for transport protocols become increasingly stringent, future versions of security protocols will prevent the firewall from obtaining the session key for data transmission, thus hindering decryption of the transmitted content and application-layer semantic analysis. Therefore, this method is ill-suited for future transport protocols.

[0092] In another alternative implementation, for transmission scenarios where middleware exists, a middlebox security protocol (MSP) is proposed. MSP can include McTLS, MaTLS, etc.

[0093] The basic principle of McTLS is to encrypt different parts of the transmitted content using different keys. Different levels of permissions can be assigned to different middleware, enabling them to perform corresponding actions. For example, assigning a middleware an encryption key grants it the permission to read the transmitted content; assigning both an encryption key and a digest key grants it the permission to read and modify the transmitted content. It should be understood that the permissions of the middleware can be negotiated and determined by the sender and receiver, who are aware of the permissions granted to each middleware. When the transmitted content is modified, the receiver can determine the modification based on the keys assigned to different middleware, thus confirming that the modification was made by a middleware with the appropriate modification permissions.

[0094] For example, such as Figure 4 As shown, assuming the transmitted content can be divided into three blocks, A, B, and C, and the transmission process includes two middlewares: MB1 and MB2. Permissions can be assigned to each middleware for each block. The permission type for a middleware can be: read permission... Figure 4 In Chinese, R represents write permission. Figure 4 (represented by W in Chinese), or no permission ( Figure 4 (Used as "—" in Chinese). Write permission is the same as modify permission. Figure 4 Figure (a) shows a diagram illustrating the allocation of permissions for blocks A, B, and C to MB1 and MB2, as follows: Figure 4As shown in (a), MB1 has read permissions (R) for blocks A, B, and C, while MB2 has read permissions (R) for block A, write permissions (W) for block B, and no permissions (—) for block C. The final allocated permissions can be as follows: Figure 4 As shown in (b), for block A, both MB1 and MB2 have read permission (R); for block B, MB1 has read permission (R) and MB2 has write permission (W); for block C, MB1 has read permission (R) and MB2 has no permission (—).

[0095] It should be understood that while auditing transmitted content based on McTLS can identify the middleware that modified the content as having the necessary permissions, if multiple middleware have the same permissions, the receiving end still cannot determine which specific middleware made the modification. Furthermore, assigning different permissions to different middleware is quite complex, and the number of keys required is directly proportional to the number of data blocks and the number of middleware.

[0096] The basic principle of MaTLS is that the sender and receiver negotiate to assign a digest key to each middleware. The middleware can generate a MAC value for the content transmitted into the middleware, and the middleware can also generate a MAC value when outputting the content. These MAC values ​​can be sent to the receiver along with the content, so that the receiver can audit the modifications and modification locations of the content on the transmission path based on the MAC values.

[0097] For example, Figure 5 For an end-to-end transmission scenario, such as Figure 5 As shown, the sender and receiver can be connected via two middleware modules: MB1 and MB2. According to MaTLS, MB1 generates MAC1 upon receiving the transmitted content and MAC2 upon sending the transmitted content. MB2 generates MAC3 upon receiving the transmitted content and MAC4 upon sending the transmitted content. The receiver can use these MAC addresses to determine any modifications to the transmitted content and the location of the modified transmission.

[0098] It should be understood that in MaTLS-based auditing methods, a key needs to be allocated to each middleware, and each middleware needs to generate a MAC address. Key allocation and MAC transmission consume significant communication resources, and MAC calculation also consumes the middleware's computational resources. For transmission scenarios with a large number of middleware, this method consumes a large amount of computational and communication resources, making it impractical.

[0099] To address the problems of the aforementioned methods, this application provides a data processing method. The dataset generated by the generator can include multiple data blocks. The generator assigns a different counting sequence to each data block and generates verification information for each data block based on the counting sequence. This ensures that the verification information of the data blocks is difficult to forge. Furthermore, the generator can map the verification information of all data blocks to a first mapping sequence, allowing the verifier to verify based on the first mapping sequence rather than on the verification information of each individual data block, thus saving storage and communication resources. The updater can generate operation record blocks to indicate the update operation for the executed update action, and the updater generates a new mapping sequence corresponding to the verification information for newly added or modified data blocks. The verifier can verify data blocks without update records based on the first mapping sequence, and can also verify data blocks added or modified by the updater based on the mapping sequence generated by the updater. It can be seen that this method can achieve fine-grained auditing. Moreover, when the updater exists, the location of the updated data block and the stage of the update can be determined. Additionally, since the verification information is generated in conjunction with the counting sequence, this method can greatly improve the unforgeability of the verification information and prevent illegal attacks. Furthermore, since the verifier can correctly verify data blocks without operation records and data blocks that have been operated on by the updater, the verifier can also detect illegal modifications without operation records, which can greatly improve the accuracy of auditing.

[0100] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. "And / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. Meanwhile, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present related concepts in a concrete manner for ease of understanding.

[0101] First, a brief introduction to the application scenarios of the data processing method provided in this application will be given. Figure 6 This is a schematic diagram of a communication system, such as... Figure 6As shown, the communication system 60 may include a generator 601 and a verifier 602, wherein the generator 601 is used to generate data, and the verifier 602 is used to verify the data. It should be understood that the generator and the verifier may be the same device or different devices; this application does not limit this. For example, taking the application scenario of auditing data in a database as an example, both the generator 601 and the verifier 602 may be database administrators (devices with database operation permissions). After storing data in the database, the database administrator can subsequently verify the stored data to check whether other users have modified the data in the database. Alternatively, as another example, taking the application scenario of end-to-end data transmission as an example, the generator 601 may be the data sender, and the verifier 602 may be the data receiver. The receiver can verify the received data to check whether the data sent by the sender has been modified.

[0102] Optionally, the communication system 60 may further include an updater 603, which can update the data generated by the generator 601. For example, taking the application scenario of auditing data in a database as an example, the updater 603 could also be a database administrator, who can modify previously stored data after storing it. As another example, taking the application scenario of end-to-end data transmission as an example, middleware (MB) can participate in the data transmission process, and the updater 603 can be the middleware performing the update operation.

[0103] It should be understood that the business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0104] Optionally, the generator, verifier, and updater in the embodiments of this application may also be referred to as data processing devices, which may be general-purpose devices or special-purpose devices. The embodiments of this application do not specifically limit this.

[0105] Optionally, the functions of the generator, verifier, and updater in the embodiments of this application can be implemented by one device, multiple devices, or one or more functional modules within a single device. This application does not specifically limit these functions. It is understood that the aforementioned functions can be network elements in hardware devices, software functions running on dedicated hardware, or a combination of hardware and software.

[0106] For example, the functions of the generator, verifier, and updater in the embodiments of this application can be achieved through... Figure 7 The data processing device 70 in the middle is used to achieve this. Figure 7 The diagram shown is a structural schematic of a data processing apparatus 70 provided in an embodiment of this application. The data processing apparatus 70 includes one or more processors 701, a communication line 702, and at least one communication interface. Figure 7 (This is merely an example illustration, using a communication interface 703 and a processor 701 as examples.) Optionally, a memory 704 may also be included.

[0107] The processor 701 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0108] Communication line 702 may include a path for connecting different components.

[0109] The communication interface 703 can be a transceiver module used to communicate with other devices or communication networks, such as Ethernet, RAN, and wireless local area networks (WLAN). For example, the transceiver module can be a transceiver or similar device. Optionally, the communication interface 703 can also be a transceiver circuit located within the processor 701, used to implement the processor's signal input and signal output.

[0110] The memory 704 can be a device with storage functionality. For example, it can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; electrically erasable programmable read-only memory (EEPROM); compact disc read-only memory (CD-ROM) or other optical disc storage; optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.); magnetic disk storage media or other magnetic storage devices; or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited to these. The memory can exist independently and be connected to the processor via communication line 702. The memory can also be integrated with the processor.

[0111] The memory 704 stores computer execution instructions for implementing the scheme of this application, and its execution is controlled by the processor 701. The processor 701 executes the computer execution instructions stored in the memory 704, thereby implementing the message forwarding method provided in the embodiments of this application.

[0112] Alternatively, in this embodiment, the processor 701 may execute the processing-related functions in the message forwarding method provided in the following embodiments of this application, and the communication interface 703 may be responsible for communicating with other devices or communication networks. This embodiment does not specifically limit this.

[0113] Optionally, the computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.

[0114] In a specific implementation, as one example, the processor 701 may include one or more CPUs, for example... Figure 7 CPU0 and CPU1 in the CPU.

[0115] In a specific implementation, as one embodiment, the data processing device 70 may include multiple processors, for example... Figure 7 The processors 701 and 707 are described herein. Each of these processors may be a single-core processor or a multi-core processor. The processors herein may include, but are not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, and other computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing.

[0116] In a specific implementation, as one embodiment, the data processing device 70 may further include an output device 705 and an input device 706. The output device 705 communicates with the processor 701 and can display information in various ways. For example, the output device 705 may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device 706 communicates with the processor 701 and can receive user input in various ways. For example, the input device 706 may be a mouse, keyboard, touchscreen device, or sensing device, etc.

[0117] The aforementioned data processing device 70 may sometimes be referred to as a data processing apparatus, which can be a general-purpose device or a special-purpose device. For example, the data processing device 70 may be a desktop computer, a portable computer, a network device, a handheld computer (PDA), a mobile phone, a tablet computer, a wireless terminal device, an embedded device, the aforementioned terminal device, the aforementioned network device, or a device with... Figure 7 Devices with similar structures. This application does not limit the type of data processing device 70 to any particular embodiment.

[0118] The following will combine Figures 2 to 7 The data processing method provided in the embodiments of this application will be described in detail.

[0119] Figure 8 This is a flowchart illustrating a data processing method provided in an embodiment of this application. This method can be applied to the data generator. Figure 8 As shown, the method may include the following steps.

[0120] Step 101: Obtain N data blocks, where N is a positive integer.

[0121] The generator can divide the data to be generated into N data blocks, so that each of the N data blocks can be verified separately, thereby achieving fine-grained auditing. Optionally, the audit granularity can be the same as the data block size, or an integer multiple of the data block size.

[0122] Optionally, the N data blocks can be plaintext or ciphertext blocks. For example, if the auditing method is executed in a trusted environment, the data block can be a plaintext block. For instance, when the auditing method is applied to auditing data in a database, and the database is a trusted environment, the N plaintext blocks can be directly divided based on the plaintext data stored in the database. If the auditing method is executed in an untrusted environment, the data block can be a ciphertext block. For instance, in end-to-end data transmission scenarios where the data transmission is not in a trusted environment, the data to be transmitted can be divided into N ciphertext blocks.

[0123] It should be understood that encrypting data into ciphertext blocks can improve data security. Conversely, dividing data into plaintext blocks without further encrypting them into ciphertext blocks can reduce processing complexity. These two methods can be chosen based on actual security needs, and this application does not limit the choice.

[0124] Optionally, if the data block is a ciphertext block, the ciphertext block can be obtained by encrypting plaintext blocks. For example, the generated data includes N plaintext blocks, and then the N plaintext blocks are encrypted to obtain N ciphertext blocks. Optionally, the N plaintext blocks can be encrypted using the same encryption method and key. It should be understood that this application does not limit the encryption method used when encrypting plaintext blocks into ciphertext blocks. For example, it can be an encryption algorithm in the Data Encryption Standard (DES) or an encryption algorithm in the Advanced Encryption Standard (AES). For example, the encryption algorithm used in the embodiments of this application can be the AES-GCM algorithm, where GCM refers to Galois / counter mode. Alternatively, it can be the ChaCha20-Poly1305 algorithm.

[0125] Alternatively, if the data block is a ciphertext block, the size of the ciphertext block will vary depending on the encryption algorithm used to determine the ciphertext block from the plaintext block. For example, the ciphertext block generated using the AES-GCM algorithm is 128 bits in size, while the ciphertext block generated using the ChaCha20-Poly1305 algorithm is 512 bits in size.

[0126] Step 102: Determine the first dataset, which includes N data blocks and a first mapping sequence.

[0127] It should be understood that determining the first dataset can be interpreted as generating the first dataset.

[0128] The verification information of the nth data block can correspond to at least one specific element in the first mapping sequence. It should be understood that, since n is a positive integer from 1 to N, the first mapping sequence can include multiple specific elements corresponding to the verification information of the N data blocks. The first mapping sequence in the first dataset can be used to verify whether the nth data block has been modified, where n is a positive integer from 1 to N.

[0129] It should be understood that when verifying a data block, the verifier can generate verification information for that data block and map it using the same mapping method as the generator. If the elements mapped by the regenerated verification information are included in multiple specific elements corresponding to the verification information of N data blocks in the first mapping sequence, it indicates that the data block to be verified belongs to the N data blocks generated by the generator. It can be seen that this method can be used to audit the data blocks included in the first dataset.

[0130] Furthermore, the verification information for the nth data block can be determined based on the nth data block and its corresponding counting sequence. Different data blocks correspond to different counting sequences.

[0131] It should be understood that since different data blocks correspond to different counting sequences, combining the counting sequences of different data blocks to determine their verification information ensures that the verification information for different data blocks is different. First, if the content of different data blocks is different, their corresponding verification information will be different. Furthermore, even if the content of different data blocks is the same, their verification information will still be different because their corresponding counting sequences are different. This method ensures that the verification information of data blocks is difficult to forge, preventing data deception.

[0132] Based on this scheme, if a data block in the first dataset is modified, the verification information corresponding to the modified data block will also change. Furthermore, since the modifier cannot know the counting sequence corresponding to the modified data block, it cannot forge the verification information of the modified data block and map it into the first mapping sequence. Therefore, the modification of the data block cannot be concealed and will definitely be detected by the verifier. Alternatively, if the modifier adds a new data block to the first dataset, since different data blocks correspond to different counting sequences, the addition of the new data block will cause changes to the counting sequences of some data blocks, thus changing the verification information of those data blocks. This behavior will be detected by the verifier. Furthermore, since the counting sequence corresponding to the newly added data block cannot be known, and the verification information of the newly added data block cannot be forged and mapped into the first mapping sequence, the newly added data block will also be detected. Alternatively, if a data block in the first dataset is deleted, since the counting sequence is based on the positional relationship corresponding to the data block, the deletion of the data block will also cause changes to the counting sequences of some data blocks, thus changing the verification information of those data blocks. This behavior will be detected by the verifier. It can be seen that the verification information generated from the counting sequence can ensure that any operation on the data block can be detected, thus preventing data deception.

[0133] In summary, this application provides a data processing method applicable to data generators. In this method, the generator can acquire multiple data blocks and generate a first dataset based on these data blocks. This first dataset includes not only multiple data blocks but also a first mapping sequence. The verification information for each data block in the first dataset corresponds to at least one specific element in the first mapping sequence, thus the first mapping sequence can be used to verify whether a data block in the first dataset has been modified. Furthermore, the verification information for each data block is generated based on each data block and its corresponding counting sequence, making the verification information difficult to forge. Based on the scheme of this application, the audit granularity can be set to be consistent with the data blocks, thereby achieving fine-grained auditing. Moreover, this application does not directly use the verification information of each data block for verification but maps it to the first mapping sequence, which can reduce the storage and communication resources required. In addition, this application combines the verification information generated with the counting sequence corresponding to the data block, which can effectively prevent data deception and ensure the reliability of the audit results.

[0134] Optionally, the audit granularity of the data processing method provided in this application can be the same as the size of the data block, or it can be an integer multiple of the size of the data block.

[0135] Optionally, in step 101 above, if the data block obtained by the generator is a ciphertext block, the generator can also determine the ciphertext block based on the plaintext block and the corresponding counting sequence. For example, it can first perform an encryption operation based on the key and the counting sequence corresponding to the plaintext block to obtain an encryption sequence, and then perform an operation on the plaintext block and the encryption sequence of its corresponding counting sequence to obtain the ciphertext block.

[0136] As one possible implementation, the process of encrypting plaintext blocks into ciphertext blocks can be performed in counter (CTR) mode. For example... Figure 9 As shown, in CTR mode, each plaintext block corresponds to a counter that is incremented sequentially. The value of the counter is the counting sequence corresponding to each plaintext block. The counting sequence corresponding to each plaintext block can be encrypted using a key to obtain the encrypted sequence corresponding to each plaintext block. Then, the plaintext block and its corresponding encrypted sequence are XORed to obtain the ciphertext block.

[0137] Optionally, the counting sequence corresponding to the plaintext block can be an initialization vector (IV).

[0138] Optionally, in step 102 above, one possible way to determine the verification information corresponding to the nth data block is as follows: First, determine the digest of the counting sequence corresponding to the nth data block based on the first key. Then, perform calculations on the digest of the counting sequence corresponding to the nth data block and the nth data block to obtain the verification information of the nth data block.

[0139] Optionally, the digest of the counting sequence corresponding to the nth data block determined by the first key can be an encrypted sequence obtained by performing encryption operations based on the first key and the counting sequence corresponding to the nth data block. It should be understood that the embodiments of this application do not limit the encryption operation used to generate the digest of the counting sequence; for example, commonly used encryption algorithms or hash algorithms can be used for encryption.

[0140] As one possible implementation, the digest of the counting sequence can be the MAC of the counting sequence, and the first key is the key corresponding to the keyed hash function used to generate the MAC.

[0141] Optionally, the length of the summary of the counting sequence can be generated using different types of algorithms based on the fine-grained auditing requirements of the upper-layer business. Different types of algorithms generate summary lengths differently. This application does not impose any limitations on this.

[0142] Optionally, this application does not limit the operation method used in obtaining the verification information of the nth data block by operating on the digest of the counting sequence corresponding to the nth data block and the nth data block. For example, the operation can be an encryption operation (such as an encryption algorithm) or a computer logic operation (such as an XOR operation).

[0143] As one possible implementation, the process of determining the verification information of the nth data block based on the nth data block and its corresponding counting sequence can also be performed in CTR mode. For example... Figure 10 As shown, in CTR mode, each data block corresponds to a progressively incrementing counter, and the counter value is the counting sequence corresponding to each data block. A digest of the counting sequence for each data block can be obtained by using the first key and the counting sequence. Then, the verification information of the data block can be obtained by XORing the data block with the digest of its corresponding counting sequence.

[0144] Optionally, the counting sequence corresponding to the data block in this embodiment can increment as the number of data blocks increases. As one possible implementation, an initial counting sequence can be set, and then the counting sequence corresponding to each data block can be determined according to the position order of the data blocks in the first dataset. The counting sequences corresponding to different data blocks increment from the initial counting sequence. It should be understood that the initial counting sequence is the counting sequence corresponding to the first data block among the N data blocks included in the first dataset.

[0145] Optionally, the first mapping sequence can be implemented based on a Bloom filter. As one possible implementation, the verification information of the nth data block can be mapped to a Bloom filter using a set of hash functions to obtain an intermediate mapping sequence, and then the first mapping sequence can be determined based on this intermediate mapping sequence. This set of hash functions can include at least one hash function. It should be understood that the verification information of the aforementioned N data blocks can be mapped using the same set of hash functions.

[0146] Taking a hash function set containing x (x is a positive integer) hash functions as an example, the process of mapping verification information to a Bloom filter through the hash function set can include: processing the verification information of the nth data block using x hash functions to obtain x hash values; then setting the x bits corresponding to these x hash values ​​in the Bloom filter as valid bits (e.g., bits with a value of 1). Thus, the verification information of the nth data block corresponds to x specific valid bits in the Bloom filter, and the verification information of all N data blocks can be mapped to the same Bloom filter. The Bloom filter after mapping the verification information of N data blocks can be considered as the aforementioned intermediate mapping sequence.

[0147] Optionally, the specific elements corresponding to the verification information of different data blocks may be different. However, when the verification information of a data block corresponds to multiple specific elements, some elements may be the same among the multiple specific elements corresponding to the verification information of different data blocks.

[0148] For example, taking mapping based on a Bloom filter as an example, assume N equals 2 and x equals 3. Figure 11As shown, a Bloom filter can include 16 bits, each initially set to 0. The verification information (verification information_1) of the first data block can be mapped to the 3rd, 4th, and 7th bits using hash functions 1, 2, and 3 respectively, setting the 3rd, 4th, and 7th bits to 1. Similarly, the verification information (verification information_2) of the second data block can be mapped to the 6th, 7th, and 10th bits using hash functions 1, 2, and 3 respectively, setting the 6th, 7th, and 10th bits to 1. It can be seen that the bits corresponding to the verification information of the first data block and the bits corresponding to the verification information of the second data block can have a duplicate bit (i.e., the 7th bit).

[0149] Optionally, the hash functions in the hash function set can be either keyless or keyed hash functions; this application does not impose any limitation on this. For example, if the auditing method is executed in a trusted environment, a keyless hash function can be used to reduce the complexity of the mapping process. If the auditing method is executed in an untrusted environment, a keyed hash function can be used to improve security.

[0150] Using a keyed hash function can increase the randomness of the mapping relationship, making both the input and output difficult to predict. This increases the complexity and difficulty for attackers to predict the hash function used in the mapping by using the input (such as verification information) and output (such as elements in the first mapping sequence). After modifying the data block, attackers can use the hash function to forge the mapped elements in the first mapping sequence, thereby improving the security of the mapping and reducing the risk of being attacked.

[0151] It should be understood that whether the hash functions in the hash function set are keyed hash functions can be selected according to actual security needs, and this application does not impose any restrictions on this.

[0152] Optionally, the intermediate mapping sequence can be directly used as the first mapping sequence in the first dataset. Alternatively, the intermediate mapping sequence can be encrypted using the second key to obtain the first mapping sequence. For example, if the auditing method is executed in a trusted environment, the intermediate mapping sequence can be directly used as the first mapping sequence in the first dataset. If the auditing method is executed in an untrusted environment, the intermediate mapping sequence can be encrypted using the second key to obtain the first mapping sequence. Encrypting the intermediate mapping sequence ensures that the mapping result corresponding to the verification information of the data block is encrypted, further preventing attackers from using the mapping result to reverse-engineer the mapping rules and preventing attackers from modifying the mapping result to conceal modifications to the data block.

[0153] Optionally, the generator may also determine a summary of the first mapping sequence, and the first dataset may also include the summary of the first mapping sequence. The summary of the first mapping sequence can be used to audit the integrity of the first mapping sequence. When verifying a data block, the verifier may regenerate the summary of the first mapping sequence and compare the regenerated summary with the original summary to determine whether the first mapping sequence has been modified.

[0154] As one possible implementation, the summary of the first mapping sequence can be the MAC of the first mapping sequence.

[0155] It should be noted that encrypting the intermediate mapping sequence to obtain the first mapping sequence, and calculating the digest of the first mapping sequence, can be regarded as protecting the confidentiality and integrity of the intermediate mapping sequence.

[0156] For example, taking data blocks as ciphertext groups, and the first mapping sequence obtained based on a Bloom filter mapping, the process of generating the first mapping sequence of the data blocks is described below. Figure 12 As shown, firstly, it consists of N plaintext blocks ( Figure 12 Plaintext blocks 1 to N are encrypted to obtain N ciphertext blocks. Figure 12 The N ciphertext blocks are _1 to _N. Then, the digest of each ciphertext block and its corresponding counting sequence is processed to obtain the verification information corresponding to the N ciphertext blocks. Figure 12 The verification information is divided into N values ​​(e.g., verification information _1 to verification information _N). Then, these N verification information values ​​are mapped to a Bloom filter using a set of hash functions with a key (the mapped Bloom filter can be considered an intermediate mapping sequence). Finally, the mapped Bloom filter is protected for confidentiality and integrity to obtain the first mapping sequence.

[0157] Optionally, the first dataset may include a count sequence corresponding to each of the N data blocks; or, the first dataset may include a starting count sequence corresponding to the N data blocks (i.e., the count sequence corresponding to the first data block among the N data blocks). This count sequence can be used to allow the verifier to regenerate the verification information corresponding to each data block based on the count sequence corresponding to each data block. It should be understood that if the first dataset includes a starting count sequence corresponding to the N data blocks, the verifier can determine the count sequence corresponding to each data block in a specific order (e.g., positional order).

[0158] Optionally, Figure 8 The data processing method shown can be applied in data transmission scenarios, where the generator is the sender of the data transmission. Optionally, in this scenario, the data processing method may further include: step 103, sending the first dataset.

[0159] Optionally, in the data transmission scenario, the key used by the sending end to generate data blocks and data block verification information can be negotiated between the sending end and the receiving end.

[0160] Optionally, in the data transmission scenario, when sending the first dataset, the sending end can also send cryptographic information related to the first dataset. The cryptographic information may include the indexes of different keys used in various encryption processes, the suite description of the encryption algorithms used, the IV of the plaintext blocks, and the counting sequence corresponding to the data blocks, etc. The cryptographic information can be used to enable the receiving end to retrieve the correct algorithm and use the correct decryption and verification parameters.

[0161] Optionally, in data transmission scenarios, in order to be compatible with existing transmission protocols, the verification information corresponding to the data block can also be carried in the message.

[0162] Optionally, in data transmission scenarios, when the sender transmits the first dataset, different contents of the first dataset can be included in different fields of the message. As a possible implementation, cryptographically related information in the first dataset can be placed in the same field (e.g., the message header), data blocks and their verification information can be placed in the same field (e.g., the payload), and the first mapping sequence and its digest can be placed in the same field (e.g., the message extension field). Placing the first mapping sequence and its digest in the message extension field improves message compatibility and prevents the message from being unrecognizable.

[0163] For example, taking a TLS message as an example, such as Figure 13 As shown, a TLS message can include a TLS header, a TLS payload field, and a TLS extension field. The TLS header can carry cryptographic information (such as a description of the cryptographic suite, key index, initialization vector IV, and counting sequence), the TLS payload field can carry data blocks and their verification information, and the TLS extension field can carry a first mapping sequence and its digest.

[0164] It should be understood that, in data transmission scenarios, the process by which the sending end (i.e., the aforementioned generator) generates the first dataset is the same as the process by which the sending end constructs a data packet. Furthermore, the data to be transmitted by the sending end is generally plaintext data; the sending end needs to encrypt the plaintext data to obtain ciphertext data to be carried in the packet.

[0165] Optionally, since the data to be transmitted is encrypted and carried in the payload field, while the generated first mapping sequence is carried in the extended field of the message, the data encryption process and the acquisition process of the first mapping sequence can be carried in parallel.

[0166] For example, Figure 14 This diagram illustrates how a sender generates a TLS message. The encryption algorithm used by the sender can be AES-GCM. The encryption of plaintext blocks into ciphertext blocks and the generation of corresponding verification information for the ciphertext blocks can both operate in CTR mode. Figure 14 As shown, the data to be transmitted can be divided into N plaintext packets (plaintext packet_1 to plaintext packet_N). The initialization vector IV (IV_1 to IV_N) corresponding to each plaintext packet can be encrypted using key K1. Then, the encrypted value is XORed with the plaintext packet to obtain the corresponding ciphertext packet, thus obtaining N ciphertext packets (ciphertext packet_1 to ciphertext packet_N). These N ciphertext packets can be concatenated to obtain ciphertext data, which can be carried in the TLS payload field. Furthermore, a digest of the ciphertext data can be calculated using key K2 through an encryption algorithm, and this digest can also be carried in the TLS payload field. Additionally, for the previously generated N ciphertext packets, a corresponding counter (CTR_1 to CTR_N) can be assigned to each ciphertext packet. The digest of the count sequence for each counter (CTR_1 digest to CTR_N digest) can be calculated using key K3. Then, an XOR operation is performed between the digest of the count sequence and its corresponding ciphertext block to obtain the verification information for each ciphertext block, resulting in N verification messages (verification message_1 to verification message_N). These N verification messages can be mapped to a Bloom filter using a set of hash functions with a key. The mapped Bloom filter (i.e., the intermediate mapping sequence) can be encrypted using key K4 to obtain the first mapping sequence, which can be carried in the TLS extension field. Furthermore, the digest of the first mapping sequence can be calculated using a cryptographic algorithm using key K5, and this digest can also be carried in the TLS extension field. Based on these steps, the sender can construct the payload field and extension field of the TLS message.

[0167] Figure 15 A flowchart illustrating another data processing method provided in this application embodiment, which can be applied to data updating. Figure 15 As shown, the method may include the following steps.

[0168] Step 201: Obtain the second dataset.

[0169] It should be understood that this second dataset is the dataset that the updater will update.

[0170] Optionally, in data transmission scenarios, the data updater can be middleware that modifies the data during data transmission. In data transmission scenarios, the updater obtaining the second dataset essentially means: the updater receives the second dataset. The second dataset received by the updater can come from the sender or from the previous hop middleware on the data transmission path.

[0171] Optionally, the first dataset generated by the generator can be updated multiple times by multiple updaters, and the updater used to obtain the second dataset can be one of these multiple updaters. It should be understood that if the updater is the first updater to update the first dataset, then the second dataset is identical to the first dataset. If the updater is not the first updater to update the first dataset, then the obtained second dataset can be a dataset after other updaters have updated the first dataset at least once.

[0172] Step 202: Update the second dataset to obtain the updated second dataset. The second dataset and the updated second dataset have at least one different data block.

[0173] It should be understood that the second dataset has at least one different data block from the updated second dataset, indicating that the updating party can update the data block in the second dataset.

[0174] Optionally, the update operation performed by the updater on the data blocks in the second dataset may include at least one of the following: adding a data block, deleting a data block, and modifying a data block.

[0175] Step 203: Determine the third dataset, which includes the updated second dataset and the operation record block. The operation record block is used to instruct the updater to update the second dataset.

[0176] It should be understood that the updating party generates an operation record block to indicate its updates to the second dataset, so that the validator can determine the specific circumstances of the data update based on the operation record block generated by the updating party when auditing the data.

[0177] In summary, this application provides a data processing method applicable to the updating party. After updating the second dataset, the updating party can generate an operation record block that instructs the updating party to update the second dataset. Therefore, during data auditing, if the verifying party discovers data changes, it can query the operation record block generated by the updating party to determine the specific details of the data update. Compared to existing auditing methods, the data processing method provided in this application allows for more detailed auditing.

[0178] Based on this scheme, the verifier can determine whether the data modification was performed by an authorized party based on the operation log block, thereby determining whether the verified data is trustworthy. It should be understood that if an unauthorized party modifies the data and does not generate an operation log block, the verifier will be unable to find the corresponding operation log block when verifying the data, thus confirming that the data was modified by an unauthorized party and is untrustworthy. Alternatively, if an authorized party performs an operation outside their authorized scope, the verifier can also determine this based on the operation log block.

[0179] Optionally, the operation record block may include location information, which can be used to indicate the location where the update occurred. As one possible implementation, the location where the update occurred can be indicated using the data block's position number. This application embodiment uses the data block's position number to describe the data block's location information as an example, and this will be explained uniformly here.

[0180] Optionally, if the operation of updating the second dataset includes adding a first data block to the second dataset, then the operation record block may include first location information, which can be used to indicate the position of the first data block in the third dataset, and can also be used to indicate the position of the update (i.e., the added data block) relative to the second dataset.

[0181] For example, such as Figure 16 As shown, assuming the second dataset can include 4 data blocks, the update operation on the second dataset includes adding one data block between the 2nd and 3rd data blocks in the second dataset. Then, the third dataset includes 5 data blocks, with the newly added data block being the 3rd data block in the third dataset. Therefore, the first position information generated by the updater can indicate that the position number of the newly added data block in the third dataset is 3, and can also indicate that the newly added data block was added between the 2nd and 3rd data blocks in the second dataset.

[0182] Optionally, for the newly added first data block, the updater can allocate a new counting sequence to the first data block, the newly allocated counting sequence being different from the counting sequences allocated by the generator for the N data blocks. Furthermore, the updater can determine the verification information of the first data block based on the counting sequence corresponding to the first data block and the first data block itself, and map the verification information of the first data block to obtain a second mapping sequence, where the verification information of the first data block corresponds to at least one specific element in the second mapping sequence.

[0183] It should be noted that the process of determining the verification information of the first data block based on the counting sequence corresponding to the first data block and the first data block, as well as the process of determining the second mapping sequence based on the verification information of the first data block, can all refer to the process of generating the verification information of the data block and the process of generating the first mapping sequence in step 102 above, and will not be repeated here.

[0184] Optionally, the counting sequence and the second mapping sequence corresponding to the first data block may be included in the operation record block.

[0185] Optionally, if the operation of updating the second dataset includes modifying a second data block within the second dataset, then the operation record block may include second location information. This second location information can be used to indicate the position of the second data block within the third dataset, and the position of the updated (i.e., modified) data block relative to the second dataset. Modifying the second data block may include at least one of the following: replacing part or all of the content of the second data block, adding content to the second data block, or deleting part of the content of the second data block.

[0186] For example, such as Figure 17 As shown, assuming the second dataset may include 5 data blocks, the update operation of the updater on the second dataset includes modifying the 3rd data block of the second dataset. Therefore, the second position information generated by the updater can indicate that the position number of the modified data block in the third dataset is 3, and can also indicate that the modified data block is located between the 2nd and 4th data blocks in the second dataset.

[0187] Optionally, the updater can assign a new counting sequence to the modified second data block, the newly assigned counting sequence corresponding to the modified second data block being different from the counting sequence assigned by the generator to the N data blocks. Furthermore, the updater can determine the verification information of the modified second data block based on the newly assigned counting sequence and the modified second data block, and map the verification information of the modified second data block to obtain a third mapping sequence, where the verification information of the modified second data block corresponds to at least one specific element in the third mapping sequence.

[0188] It should be noted that the process of determining the verification information of the modified second data block based on the corresponding counting sequence and the modified second data block, as well as the process of determining the third mapping sequence based on the verification information of the modified second data block, can all refer to the process of generating the verification information of the data block in step 102 above, and the process of generating the first mapping sequence, which will not be repeated here.

[0189] Optionally, the newly allocated counting sequence and the third mapping sequence of the modified second data block can be included in the operation record block.

[0190] Optionally, if the operation of updating the second dataset includes deleting a third data block from the second dataset, then the operation record block may include third location information, which can be used to indicate the position of the updated (i.e., deleted data block) relative to the second dataset.

[0191] For example, for example, such as Figure 18 As shown, assuming the second dataset can include 5 data blocks, the update operation on the second dataset can include deleting the 3rd data block. Then, the third dataset will include 4 data blocks. The third location information generated by the updater can indicate that the deleted data block is located between the 2nd and 4th data blocks in the second dataset.

[0192] As can be seen from the implementation methods corresponding to the different update operations described above, when the updater updates the second dataset, the location information can indicate the position of the update (addition, modification, or deletion) relative to the second dataset. Furthermore, if the update operation is to add or modify a data block, the location information can also indicate the position of the added or modified data block within the third dataset.

[0193] It should be understood that the location information in the operation log block enables the verifier to identify the data blocks generated by the producer and the updater from multiple data blocks, allowing the verifier to correctly verify each data block, thus meeting the requirements of fine-grained auditing.

[0194] In addition, the mapping sequence (the second mapping sequence and the third mapping sequence mentioned above) that generates verification information for newly added or modified data blocks can be used to verify whether the data blocks generated by the updater have been modified subsequently.

[0195] Optionally, when assigning a new counting sequence to a newly added or modified data block, a different starting counting sequence can be used than that of the generator. For example, if the generator can start assigning a counting sequence from 1 when assigning a counting sequence to the generated data block, then the updater can start assigning a new counting sequence from 1000 when assigning a new counting sequence to the newly added or modified data block.

[0196] As one possible implementation, the position information in the operation record block may include: the start and end points of the current data block and the start and end points of the original data block. The start and end points of the current data block can be used to indicate the position of the update in the third dataset, and the start and end points of the original data block can be used to indicate the position of the update in the second dataset. The start and end points of the data block can be represented by the position sequence number of the data block. For example, the first and second position information mentioned above may include the start and end points of the current data block and the original data block, while the third position information may only include the start and end points of the original data block.

[0197] For example, with Figure 16 Taking the update operation shown as an example, the newly added data block is located at position 3 in the third dataset. The start and end points of the existing data block in the first position information can be {3, 3}. Optionally, {3, 3} can be abbreviated to {3}. The newly added data block is added between the second and third data blocks in the second dataset, so the start and end points of the original data block in the first position information can be {2, 3}.

[0198] For example, with Figure 17 Taking the update operation shown as an example, the modified data block is located at position 3 in the third dataset. The start and end points of the current data block in the second position information can be {3, 3}, which can be abbreviated as {3}. The modified data block is also located at position 3 in the second dataset, that is, between the second and fourth data blocks. Therefore, the start and end points of the original data block in the second position information can be {2, 4}.

[0199] For example, with Figure 18 Taking the update operation shown as an example, the deleted data block is the third data block in the second dataset, that is, between the second and fourth data blocks. Therefore, the start and end points of the original data block in the third location information can be {2, 4}.

[0200] Optionally, the location information in the operation record block may include one or more entries, the number of which is related to the update operation performed by the updater. For example, if the update operation involves updating multiple data blocks, then one location information entry can be generated for each updated data block. It should be understood that the updates to these multiple data blocks can be the same type of update (either an addition, modification, or deletion) or different types of updates (including multiple types of addition, modification, and deletion).

[0201] For example, such as Figure 19 As shown, the second dataset can include four data blocks. The updater can add a new data block between the first and second data blocks of the second dataset, and can also modify the third data block of the second dataset. Therefore, the operation record block generated by the updater can include two location information entries, corresponding to the addition operation and the modification operation, respectively. Taking the location information including the start and end points of the current data block and the start and end points of the original data block as an example, as... Figure 19 As shown, in location information 1, the current data block starts and ends at {2}, and the original data block starts and ends at {1, 2}. In location information 2, the current data block starts and ends at {4}, and the original data block starts and ends at {2, 4}.

[0202] Optionally, if the updater updates multiple data blocks in the second dataset, and if the positions of the multiple data blocks are consecutive, then a single location information can be used to indicate the update positions of the multiple data blocks.

[0203] For example, such as Figure 20 As shown, the second dataset can include 4 data blocks. The updater adds 3 data blocks between the 2nd and 3rd data blocks in the second dataset, resulting in the third dataset containing 7 data blocks. Taking the location information, which includes the start and end points of the current data blocks and the start and end points of the original data blocks, as an example, since the newly added data blocks are located at positions 3, 4, and 5 in the third dataset, the start and end points of the current data blocks in the location information can be {3, 5}. Since the newly added data blocks are added between the 2nd and 3rd data blocks in the second dataset, the start and end points of the original data blocks in the location information can be {2, 3}.

[0204] For example, such as Figure 21 As shown, the second dataset can include 5 data blocks. The updater modifies and merges the 2nd to 4th data blocks in the second dataset into 2 data blocks, resulting in the third dataset containing 4 data blocks. Taking the location information, which includes the start and end points of the current data block and the start and end points of the original data block, as an example, since the 2nd and 3rd data blocks in the third dataset are located at positions 2 and 3, the start and end points of the current data block in the location information can be {2, 3}. Since the modified data block is the 2nd to 4th data block of the second dataset (located between the 1st and 5th data blocks), the start and end points of the original data block in the location information can be {1, 5}.

[0205] For example, such as Figure 22 As shown, the second dataset may include 5 data blocks. The updater deletes the 2nd and 3rd data blocks in the second dataset, so the third dataset includes 3 data blocks. Taking the location information including the start and end points of the original data blocks as an example, since the deleted data blocks are located between the 1st and 4th data blocks in the second dataset, the start and end points of the original data blocks in the location information corresponding to this update are: {1, 4}.

[0206] Optionally, the operation record block generated by the updater may further include at least one of the following: the updater's identifier, the time when the update occurred or ended, the type of update, the source of the second dataset, or a signature of the operation record block. The time when the update occurred or ended can be represented by a timestamp, the type of update can include addition, modification, or deletion, and the source of the second dataset refers to the device that generated the second dataset. It should be understood that the updater's update operation targets data blocks in the second dataset; therefore, the source of the second dataset essentially refers to the source of the data block being updated by the updater. The source of the data block can be the generator or the updater hop before the current updater.

[0207] Taking location information including the start and end points of the current data block and the start and end points of the original data block as an example, Figure 23 This is a schematic diagram of an operation record block, such as... Figure 23As shown, the operation record block may include the updater identifier, timestamp, start and end dates of the current data block, start and end dates of the original data block, source of the data block, operation type, count sequence of the target data block, mapping sequence of the target data block, and signature of the operation record block. The target data block refers to the data block added or modified by the updater during the update, and the operation type is the update type mentioned above. It should be understood that if the update operation only includes deletion, then the operation record block generated by the updater will not include: start and end dates of the current data block, count sequence of the target data block, and mapping sequence of the target data block. Furthermore, the start and end dates of the current data block, start and end dates of the original data block, source of the data block, and operation type can be considered as the updater's operation record.

[0208] Optionally, the data generated by the generator can be updated by multiple updaters. In scenarios where multiple updaters participate in data updates, subsequent updaters update based on the data updated by the previous updaters. It should be understood that the third dataset generated after the previous updater's update can serve as the second dataset before the subsequent updater's update.

[0209] For example, Figure 24 A schematic diagram illustrating an update involving multiple updaters is shown, such as... Figure 24 As shown, the dataset 'a' generated by the generator can include 4 data blocks. The updater 1 can add 2 data blocks between the 3rd and 4th data blocks in dataset 'a', resulting in dataset 'b' containing 6 data blocks. Since the newly added data blocks are the 4th and 5th data blocks in dataset 'b', the start and end points of the current data blocks in operation record block 1 generated by updater 1 can be {4, 5}. Since the newly added data blocks are inserted between the 2nd and 3rd data blocks in dataset 'a', the start and end points of the original data blocks in operation record block 1 generated by updater 1 can be {3, 4}. Furthermore, the data blocks in operation record block 1 generated by updater 1 originate from the generator, and the operation type is add. Updater 2 updates based on dataset 'b', as follows... Figure 24 As shown, updater 2 can modify the 2nd to 4th data blocks in dataset b, merging them into two data blocks to obtain the updated dataset c. The modified data blocks in dataset c are the 2nd and 3rd data blocks, therefore the start and end points of the current data blocks in operation record block 2 generated by updater 2 can be {2, 3}. Since the modified data blocks are the 2nd to 4th data blocks in dataset b, located between the 1st and 5th data blocks, the start and end points of the original data blocks in operation record block 2 generated by updater 2 can be {1, 5}. Furthermore, the data blocks in operation record block 2 generated by updater 2 originate from updater 1, and the operation type is modification.

[0210] Optionally, the data processing method provided in this application can be applied to data transmission scenarios, where the updater can be middleware that modifies the transmitted data. In this case, the identifier of the updater in the operation record block can be a middleware identification (MBID), and the source of the data block can be the sender or the previous hop middleware.

[0211] Optionally, in data transmission scenarios, after determining the third dataset, the middleware can also send the third dataset. It should be understood that in data transmission scenarios, the second dataset received by the middleware is equivalent to the input data packet, and the third dataset sent is equivalent to the output data packet. The operation record block generated by the middleware can be carried in the data packet and sent to the receiving end.

[0212] Optionally, in data transmission scenarios, multiple middleware can participate in modifying data packets. In this scenario, each middleware can modify the received data packet, generate a corresponding operation record block, and then append the operation record block to the modified data packet before sending it.

[0213] For example, Figure 25 This illustrates a scenario where multiple middleware components participate in data transmission, such as... Figure 25 As shown, after modifying a data packet, middleware MB1 can generate operation record block 1. Operation record block 1 can include: MBID (MB1), timestamp (T1), operation record, mapping sequence generated by MB1, and signature of the operation record block. MB1 can append this data block 1 to the sent data packet. Middleware MB2 can receive data packets from MB1, modify data packets, and generate operation record block 2. (Continue to refer to...) Figure 25 Operation record block 2 may include: MBID (MB2), timestamp (T2), operation record, mapping sequence generated by MB1, and signature of the operation record block. MB2 may append this data block 2 to its outgoing data packets. It should be understood that the dataset updated by MB2 comes from MB1, therefore the operation record in the operation record block may point to MB1.

[0214] It should be understood that the operation record block generated by the updater can be encrypted using a key to achieve confidentiality and integrity protection. For example, taking the updater as middleware, information such as MBID, timestamp, start and end of the current data block, start and end of the original data block, data block source, operation type, MB-allocated count sequence, MB-generated mapping sequence, and the signature of the operation record block in the operation record block can all be encrypted.

[0215] Optionally, when encrypting the operation record block, instead of using the same key to encrypt all information within the block, different keys are used to encrypt different information in layers. For example, encryption layers can be divided according to the importance of the information within the operation record block. It should be understood that layered encryption of the operation record block facilitates granting different levels of access permissions to different users accessing the block, thereby improving information security.

[0216] For example, such as Figure 26 As shown, the start and end of the original data block, the source of the data block, and the operation type in the operation record block can be encrypted using key 3. The counting sequence allocated by MB can be encrypted using key 2. The start and end of the current data block can be encrypted using key 1. MBID, timestamp, mapping sequence generated by MB, and signature can be encrypted using key 0.

[0217] Optionally, in this embodiment, the keys used by the generator and the updater can be pre-negotiated. As one possible implementation, a key tree can be derived based on a key index, and this key tree includes different keys. The generator and the updater can select different keys from the key tree according to different encrypted objects.

[0218] For example, such as Figure 27 As shown, the key corresponding to the ciphertext data, the key corresponding to the first mapping sequence, and the key corresponding to the operation record block can be derived based on the key index. The key corresponding to the ciphertext data may include: the key A1 used to encrypt the ciphertext block based on the plaintext block (e.g., ...). Figure 14 The key K1) and the key A2 (e.g., the key used to generate the digest of the ciphertext data) Figure 14 The key K2 in the first mapping sequence may include: the key B1 used to generate the digest of the counting sequence corresponding to the data block (e.g., the key K2 in the first mapping sequence). Figure 14 The key K3 in the middle), the key B2 for generating the hash function set of the intermediate mapping sequence, and the key B3 for encrypting the intermediate mapping sequence to generate the first mapping sequence (e.g., the ... Figure 14 The key K4 in the first mapping sequence, and the key B4 used to generate the digest of the first mapping sequence (e.g., the key K4 in the first mapping sequence), and the key B4 used to generate the digest of the first mapping sequence (e.g., the key K4 in the first mapping sequence). Figure 14 The key corresponding to the operation record block may include: the key C1 used to sign the operation record block, and the key C2 used to encrypt the information in the operation record block.

[0219] It should be understood that, because the information in the operation record block can be encrypted in layers, therefore Figure 27 The key C2 can include multiple keys at different levels. These keys at different levels can have a derivation relationship, with higher-level keys deriving lower-level keys. For example... Figure 27As shown, key C2 can include C21, C22, C23, and C24. C21 can derive C22, C22 can derive C23, and C23 can derive C24. Based on this scheme, different levels of access permissions can be defined according to different levels of keys. For example, a device that obtains a high-level key (such as C21) can derive lower-level keys (such as C22, C23, and C24) on its own, thereby gaining access to more information and having higher access permissions.

[0220] Optionally, Figure 27 The key C21 in the middle can be Figure 26 key0 in Figure 27 The key C22 in the data can be Figure 26 key1 in Figure 27 The key C23 in the data can be Figure 26 key2 in Figure 27 The key C24 in the data can be Figure 26 The key in the file is key3.

[0221] Figure 28 A flowchart illustrating another data processing method provided in this application, which can be applied to data verification. Figure 28 As shown, the method may include the following steps.

[0222] Step 301: Obtain the fourth dataset, which includes M data blocks and the first mapping sequence, where M is a positive integer.

[0223] It should be noted that the first mapping sequence in the fourth dataset is the mapping sequence determined by the generator based on the N data blocks generated by the generator. The M data blocks in the fourth dataset include some or all of the N data blocks generated by the generator.

[0224] Optionally, in the data transmission scenario, the verifier can be the data receiver, and obtaining the fourth dataset can be: receiving the fourth dataset.

[0225] Optionally, in the data transmission scenario, the fourth dataset can originate from the sender; in this case, the fourth dataset is the same as the first dataset. Alternatively, the fourth dataset may originate from middleware involved in data modification; in this case, the fourth dataset is generated by the middleware.

[0226] Step 302: Determine K data blocks out of M data blocks, where K is a positive integer less than or equal to M.

[0227] As described in the previous embodiment of the updating party, if the updating party performs an update operation, it will generate an operation record block to indicate the update that occurred. In this embodiment, the K data blocks identified refer to data blocks without update records, meaning these K data blocks do not belong to the data blocks corresponding to the update operation indicated by the operation record block.

[0228] Optionally, the fourth dataset may include J operation record blocks, each corresponding one-to-one with one of the J updaters. Each operation record block can be used to indicate the update performed on the original dataset by the updater that generated the operation record block. The original dataset is the dataset before the updater performed the update. A method for determining K data blocks from the M data blocks may include: determining K data blocks from the M data blocks based on the J operation record blocks.

[0229] As one possible implementation, determining K data blocks from M data blocks based on J operation record blocks can specifically include the following steps:

[0230] Step 3021: Based on the j-th operation record block, determine the target data block corresponding to the update data block of the j-th updater among the M data blocks.

[0231] Here, j is a variable, and its initial value is J. This application sets the initial value of j to J, indicating that the operation record blocks of each updater are verified by working backward from the last updater's operation record block. Furthermore, the updated data block refers to the data block in the second original dataset indicated by the j-th operation record block, which is obtained by the j-th updater after updating the first original dataset. It should be understood that when j = J, the fourth dataset is obtained after the j-th updater's update.

[0232] It should be noted that, as illustrated in the above embodiments for the updating party, when the updating party's update operation is to add or modify, the generated operation record block can indicate the position of the newly added or modified data block in the updated dataset (such as the second original dataset). Therefore, the updated data block of the j-th updating party refers to the data block added or modified by the j-th updating party.

[0233] Optionally, the j-th operation record block may include fourth location information, which may indicate the position of the updated data block of the j-th updater in the second original dataset, and the position of the update performed by the j-th updater relative to the first original dataset. Based on this optional implementation, the verifier can determine the updated data block of the j-th updater according to the fourth location information in the j-th operation record block.

[0234] Optionally, the verifier can determine the target data block corresponding to the updated data block of the j-th updater based on the positional relationship between the data blocks in the second original dataset and the M data blocks in the fourth dataset. It should be understood that, as shown in step 3022 below, the positional relationship between the data blocks in the second original dataset obtained after the j-th updater's update and the M data blocks in the fourth dataset can be determined by the (j+1)-th updater. This allows the j-th updater to determine the target data block corresponding to the updated data block.

[0235] It should be noted that the target data block corresponding to the updated data block of the j-th updater refers to the data block directly obtained from the updated data block of the j-th updater among the M data blocks. For example, such as... Figure 24 As shown, the updated data blocks for updater 2 are the 2nd and 3rd data blocks in dataset c. Since dataset c is the fourth dataset, the 2nd and 3rd data blocks in dataset c are the target data blocks. (Continue to refer to...) Figure 24 The updated data block of updater 1 includes the 4th and 5th data blocks in dataset b. The 5th data block in dataset b was not subsequently modified by updater 2 and directly obtained the 4th data block in dataset c. Therefore, the target data block corresponding to the updated data block of updater 1 is the 4th data block in dataset c.

[0236] It should be understood that if the update operation of the j-th updater is to delete a data block from the first original dataset, then the j-th updater does not generate a new data block after updating the first original dataset. Therefore, the updated data block of the j-th updater does not exist in the second original dataset, and consequently, there is no corresponding target data block among the M data blocks. Thus, if the update operation of the j-th updater is a deletion operation, the corresponding target data block cannot be determined based on the j-th operation record block.

[0237] Step 3022: If j-1 is not equal to 0, based on the j-th operation record block, determine the position of the data blocks in the first original dataset, excluding the target data block corresponding to the updated data block of the j-th updater. Then, decrement j by 1 and re-execute step 3021.

[0238] When j-1 is not equal to 0, it means that there are other updaters before the j-th updater.

[0239] It should be understood that after the verifier determines the target data block corresponding to the updated data block of the j-th updater, it only needs to continue to verify the other data blocks in the M data blocks. Therefore, it only needs to determine the position of the data blocks other than the target data block corresponding to the updated data block of the j-th updater in the first original dataset.

[0240] After determining the position of the data blocks in the first original dataset, excluding the target data block corresponding to the updated data block of the j-th updater, the verifier can decrement j by 1 and re-execute step 3021.

[0241] It should be understood that the first original dataset operated on by the j-th updater is generated by the (j-1)-th updater. The j-th updater determines the position of the data blocks in the first original dataset other than the target data block corresponding to the updated data block of the j-th updater. This is equivalent to determining the position of the data blocks in the dataset updated by the (j-1)-th updater, other than the target data block corresponding to the updated data block of the j-th updater. Decrementing j by 1 and re-executing step 3021 is to continue to determine whether there is a target data block corresponding to the updated data block of the (j-1)-th updater among the M data blocks.

[0242] It should be understood that if j-1 is not equal to 0, steps 3021 and 3022 will be executed repeatedly, thereby determining all target data blocks among the M data blocks that correspond to the updated data blocks of the updater.

[0243] Step 3023: If j-1 equals 0, determine the data blocks other than the target data block from the M data blocks as K data blocks.

[0244] It should be understood that when j-1 equals 0 (i.e., j=1), it indicates that the j-th updater is the first updater that updates the first dataset generated by the generator, and there are no other updaters before the j-th updater. Since steps 3021 and 3022 can determine all target data blocks in the M data blocks corresponding to the updater's updated data blocks, the data blocks in the M data blocks other than the target data blocks corresponding to the updater's updated data blocks are data blocks without update records. Therefore, the verifier can determine the data blocks in the M data blocks other than the target data blocks corresponding to the updater's updated data blocks as K data blocks.

[0245] For example, with Figure 24 The following example of the update scenario will be used to explain steps 3021 to 3023 in detail. Figure 24 The scenario involves two updaters (J=2), dataset c is the fourth dataset, and M data blocks refer to the 5 data blocks in dataset c. First, the initial value of j is 2. The verifier will first verify operation record block 2 generated by updater 2. Based on the start and end points {2, 3} of the current data blocks in the operation record block generated by updater 2, the verifier can determine that the updated data blocks of updater 2 are the 2nd and 3rd data blocks in dataset c. Since dataset c is the fourth dataset, the 2nd and 3rd data blocks in dataset c are also the target data blocks in dataset c.

[0246] Then, since there is an updater 1 before updater 2, the verifier can determine the position of the data blocks in dataset c (excluding the 2nd and 3rd data blocks, i.e., the 1st, 4th, and 5th data blocks in dataset c) in dataset b based on the start and end points {1, 5} of the original data blocks in the operation record block generated by updater 2. For example... Figure 24 As shown, the first data block in dataset c corresponds to the first data block in dataset b, the fourth data block in dataset c corresponds to the fifth data block in dataset b, and the fifth data block in dataset c corresponds to the sixth data block in dataset b.

[0247] Subsequently, the verifier can verify operation record block 1 generated by updater 1. Based on the start and end points {4, 5} of the current data blocks in the operation record block generated by updater 1, the verifier can determine that the updated data blocks of updater 1 are the 4th and 5th data blocks in dataset b. Among them, only the 5th data block in dataset b was not modified afterward and directly became the 4th data block in dataset c. Therefore, the 4th data block in dataset c is the target data block corresponding to the updated data block of updater 1.

[0248] Since there are no other updaters before verifier 1, verifier 1 can determine that the data blocks in dataset c, except for the target data blocks corresponding to the updated data blocks of the updaters (the 2nd, 3rd, and 4th data blocks in dataset c), are data blocks without update records. That is, the 1st and 5th data blocks in dataset c are data blocks without update records.

[0249] Optionally, if the fourth dataset does not include operation record blocks, then the M data blocks in the fourth dataset are all data blocks without operation records, and this also means that the N data blocks generated by the generator do not have corresponding operation records, thus K=M=N.

[0250] It should be understood that in step 302, the verifier only identified K data blocks without update records based on the operation record blocks. This does not mean that these K data blocks have truly not been modified (there may be unauthorized modifications that were not recorded). Therefore, these K data blocks need to be verified further. Theoretically, the K data blocks without update records should be generated by the generator, and these K data blocks should correspond to the data blocks generated by the generator. Thus, it can be subsequently verified whether these K data blocks belong to the N data blocks generated by the generator to determine whether these K data blocks have been modified.

[0251] Optionally, determining the K data blocks further includes determining the positions of the K data blocks in the first dataset.

[0252] It should be understood that in step 3022 above, when j-1 is not equal to 0, the verifier can determine the position of the data blocks in the first original dataset, excluding the target data block corresponding to the updated data block of the j-th updater. Since steps 3021 and 3022 can be repeated until j-1 = 0, steps 3021 and 3022 can determine the position of the data blocks in the first original dataset corresponding to the second updater (excluding the target data blocks corresponding to the updated data blocks of all updaters (from the j-th to the 1-th updater)) in the first original dataset, which is also the position in the second original dataset corresponding to the first updater. Optionally, after step 3023, the verifier can determine the position of the updated data block of the first updater in the first dataset based on the first operation record block, and thus determine the position of the K data blocks in the first dataset.

[0253] For example, continuing with Figure 24 Taking the update scenario shown as an example, after determining that the data blocks in dataset c without updated records (i.e., K data blocks out of M data blocks) are the 1st and 5th data blocks, we can determine that the 1st to 3rd data blocks in dataset b are the 1st to 3rd data blocks in dataset a, and the 6th data block in dataset b is the 4th data block in dataset a, based on the start and end points of the current data block {4, 5} and the start and end points of the original data block {3, 4} in the 1st operation record block. Since the 1st data block in dataset c corresponds to the 1st data block in dataset b, and the 5th data block in dataset c corresponds to the 6th data block in dataset b, we can determine that the 1st data block in dataset c corresponds to the 1st data block in dataset a, and the 5th data block in dataset c corresponds to the 4th data block in dataset a. Thus, the positions of the K data blocks out of M data blocks in the first dataset are determined.

[0254] Step 303: Determine the verification information of the kth data block based on the kth data block and the corresponding counting sequence, where k is a positive integer from 1 to K.

[0255] Optionally, the verifier may receive the counting sequence corresponding to the k-th data block from the generator in advance, or the verifier may obtain the counting sequence corresponding to the k-th data block from the fourth dataset.

[0256] As one possible implementation, the fourth dataset may also include an initial count sequence, which is the initial count sequence corresponding to the N data blocks generated by the generator.

[0257] Optionally, taking the fourth dataset including the initial counting sequence as an example, after determining the K data blocks in the fourth dataset, the verifier can also determine the corresponding counting sequence for each of the K data blocks based on the initial counting sequence. Since these K data blocks correspond to the data blocks in the first dataset, the corresponding counting sequences can be assigned according to the positions of these K data blocks in the first dataset, so that the K data blocks can be correctly verified.

[0258] After determining the counting sequences corresponding to the K data blocks, the verification information of the k-th data block can be determined based on the k-th data block and its corresponding counting sequence. The verification information of the k-th data block can be used to verify whether it is a data block in the first dataset generated by the generator, that is, to verify whether the k-th data block has been illegally tampered with after its generation.

[0259] It should be noted that the process of determining the verification information of the k-th data block based on the k-th data block and its corresponding counting sequence can be referenced from the process by which the generator generates the verification information of the n-th data block, and will not be elaborated here. Furthermore, it should be understood that the calculation method used to generate the verification information of the k-th data block should be the same as the calculation method used by the generator when generating the verification information of the data block corresponding to that k-th data block. Only in this way can the modification of the k-th data block be correctly verified.

[0260] Step 304: Determine the fourth mapping sequence. The verification information of the k-th data block corresponds to at least one specific element in the fourth mapping sequence.

[0261] After determining the verification information of the k-th data block, a fourth mapping sequence can be generated based on this verification information. It should be noted that the mapping method used to generate the fourth mapping sequence can be the same as the mapping method used to generate the first mapping sequence, for comparison purposes. For example, the verification information of the k-th data block can be mapped onto a Bloom filter to obtain the fourth mapping sequence.

[0262] Step 305: Determine whether the k-th data block has been modified based on the first mapping sequence and the fourth mapping sequence.

[0263] As one implementation, if at least one specific element corresponding to the verification information of the k-th data block in the fourth mapping sequence is included in at least one specific element corresponding to the verification information of the N data blocks in the first mapping sequence, then the k-th data block can be considered one of the N data blocks generated by the generator, and thus the k-th data block can be considered unmodified. If at least one specific element corresponding to the verification information of the k-th data block in the fourth mapping sequence is not included in at least one specific element corresponding to the verification information of the N data blocks in the first mapping sequence, then the k-th data block can be considered modified.

[0264] In summary, this application provides a data processing method applicable to data verification. The verifier can identify K data blocks without operation records based on operation record blocks, and generate a fourth mapping sequence mapping the verification information of these K operation record blocks in the same way as the generator. Then, based on the first and fourth mapping sequences, the verifier can determine whether the K data blocks without operation records have truly not been modified. This method processes data at the data block level, making it suitable for fine-grained auditing. Furthermore, in this method, the verifier can audit data blocks with modification records based on operation record blocks, and can also precisely audit whether data blocks without operation records have been modified based on the first and fourth mapping sequences, thus improving audit accuracy. Additionally, since the verification information is generated by combining the counting sequence of data blocks, data deception can be effectively prevented, and the error rate of the verifier's audit can be greatly reduced.

[0265] Optionally, if the updated data block of the j-th updater is a newly added or modified data block, the j-th operation record block in the fourth dataset may further include a counting sequence corresponding to the updated data block of the j-th updater and a fifth mapping sequence, wherein the verification information of the updated data block of the j-th updater corresponds to at least one specific element in the fifth mapping sequence. The fifth mapping sequence can be used to verify whether the updated data block of the j-th updater has been subsequently tampered with.

[0266] It should be understood that if the information indicated by the operation record block indicates that no other updater has updated the data block after the update of the j-th updater, then the update of the j-th updater should be the same as the target data block corresponding to the update of the j-th updater in the M datasets.

[0267] Therefore, when verifying the operation record block, the verifier can determine the verification information of the target data block corresponding to the updated data block based on the target data block and the counting sequence of the updated data block. Furthermore, the verifier can generate a sixth mapping sequence based on the verification information of the target data block, where the verification information corresponds to at least one specific element in the sixth mapping sequence. Finally, the verifier can verify whether the updated data block of the j-th updater is the same as its corresponding target data block based on the fifth and sixth mapping sequences.

[0268] If they are the same, it means that the target data block corresponding to the updated data block of the j-th updater is indeed directly obtained from the updated data block of the j-th updater, and has not been illegally tampered with in between. It should be understood that if it is verified that the updated data block of the j-th updater is different from its corresponding target data block, it means that the updated data block of the j-th updater was illegally tampered with after the j-th updater performed the update.

[0269] Based on this scheme, the verifier can also verify the target data block corresponding to the updated data block in the received dataset, thereby determining whether the target data block has been illegally tampered with without being recorded after being updated by the updater. Thus, based on the data processing method provided in this application, the verifier can verify both data blocks without operation records and data blocks with operation records, achieving verification of each data block in the received fourth dataset, realizing fine-grained and sophisticated auditing.

[0270] It should be noted that the actions of the generator in the above method embodiments can be performed by... Figure 7 The processor 701 in the data processing device 70 shown calls the application code stored in the memory 702 to instruct the generator to execute; the updater's action can be performed by... Figure 7 The processor 701 in the data processing device 70 shown calls the application code stored in the memory 702 to instruct the updater to execute; the verifier's actions can be performed by... Figure 7 The processor 701 in the data processing device 70 shown calls the application code stored in the memory 702 to instruct the verifier to execute; this embodiment does not impose any limitations on this.

[0271] It is understood that in the above embodiments, the methods and / or steps implemented by the generating party can also be implemented by components (e.g., chips or circuits) that can be used by the generating party; the methods and / or steps implemented by the updating party can also be implemented by components (e.g., chips or circuits) that can be used by the updating party; and the methods and / or steps implemented by the verifying party can also be implemented by components (e.g., chips or circuits) that can be used by the verifying party.

[0272] Optionally, embodiments of this application also provide another data processing apparatus for implementing the various methods described above. This data processing apparatus can be the generator in the above method embodiments, or an apparatus including the generator, or a component usable by the generator; or, the data processing apparatus can be the updater in the above method embodiments, or an apparatus including the updater, or a component usable by the updater; or, the data processing apparatus can be the verifier in the above method embodiments, or an apparatus including the verifier, or a component usable by the verifier. It is understood that, in order to implement the above functions, this data processing apparatus includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0273] This application embodiment can divide the data processing device into functional modules according to the above method embodiment. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0274] Figure 29 A schematic diagram of another data processing device is shown. This data processing device 290 includes a processing module 2901. Optionally, it may also include a transceiver module 2902, which can also be called a transceiver unit to implement transceiver functions; for example, it may be a transceiver circuit, transceiver, transceiver device, or communication interface.

[0275] Taking the data processing device 290 as an example of the generator in the above method embodiment:

[0276] Processing module 2901 can be used to acquire N data blocks, where N is a positive integer. Processing module 2901 can also be used to determine a first dataset, which includes N data blocks and a first mapping sequence. The verification information of the nth data block corresponds to at least one specific element in the first mapping sequence, which is used to verify whether the nth data block has been modified. The verification information of the nth data block is determined based on the nth data block and its corresponding counting sequence; different data blocks correspond to different counting sequences. n is a positive integer from 1 to N.

[0277] Optionally, the processing module 2901 can also be used to: determine the digest of the counting sequence corresponding to the nth data block based on the first key; and to perform operations on the digest of the counting sequence corresponding to the nth data block and the nth data block to obtain the verification information of the nth data block.

[0278] Optionally, the processing module 2901 can also be used to: map the verification information of the nth data block to a Bloom filter through a set of hash functions to obtain an intermediate mapping sequence, wherein the set of hash functions includes at least one hash function;

[0279] Determine the first mapping sequence based on the intermediate mapping sequence.

[0280] Optionally, when determining the first mapping sequence based on the intermediate mapping sequence, the processing module 2901 may specifically be used to: encrypt the intermediate mapping sequence according to the second key to obtain the first mapping sequence; or, determine the intermediate mapping sequence as the first mapping sequence.

[0281] Optionally, the processing module 2901 can also be used to determine a summary of the first mapping sequence, and the first dataset also includes a summary of the first mapping sequence.

[0282] Optionally, the transceiver module 2902 can also be used to send the first dataset.

[0283] Taking the data processing device 290 as an example of the updating method in the above embodiment:

[0284] Processing module 2901 can be used to acquire a second dataset. Processing module 2901 can also be used to update the second dataset to obtain an updated second dataset, wherein the updated second dataset and the second dataset have at least one different data block. Processing module 2901 can also be used to determine a third dataset, which includes the updated second dataset and an operation record block, the operation record block being used to instruct the updater on the second dataset.

[0285] Optionally, when acquiring the second dataset, the processing module 2901 can specifically be used to receive the second dataset via the transceiver module 2902.

[0286] Optionally, when updating the second dataset, the processing module 2901 may specifically be used to: add a first data block to the second dataset. The operation record block includes first location information, which indicates the position of the first data block in the third dataset and the position relative to the second dataset.

[0287] Optionally, when updating the second dataset, the processing module 2901 can specifically be used to: modify a second data block in the second dataset. Modifying the second data block includes at least one of the following: replacing part or all of the content of the second data block, adding content to the second data block, or deleting part of the content of the second data block. The operation record block includes second location information, which indicates the position of the second data block in the third dataset and updates its position relative to the second dataset.

[0288] Optionally, when updating the second dataset, the processing module 2901 may specifically be used to: delete a third data block from the second dataset. The operation record block includes third location information, which indicates the position of the update relative to the second dataset.

[0289] Optionally, after determining the third dataset, the transceiver module 2902 can also be used to send the third dataset.

[0290] In this example, the data processing device 290 is used as the verification unit in the above method embodiment:

[0291] Processing module 2901 can be used to acquire a fourth dataset, which includes M data blocks and a first mapping sequence, where M is a positive integer. Processing module 2901 can also be used to determine K data blocks from the M data blocks, where K is a positive integer less than or equal to M. Processing module 2901 can also be used to determine the verification information of the k-th data block based on the k-th data block and its corresponding counting sequence, where k is a positive integer from 1 to K. Processing module 2901 can also be used to determine a fourth mapping sequence, where the verification information of the k-th data block corresponds to at least one specific element in the fourth mapping sequence. Processing module 2901 can also determine whether the k-th data block has been modified based on the first mapping sequence and the fourth mapping sequence.

[0292] Optionally, when determining K data blocks from M data blocks based on J operation record blocks, processing module 2901 can specifically be used to: determine the target data block corresponding to the update data block of the j-th updater among the M data blocks based on the j-th operation record block. Here, the update data block is a data block in the second original dataset indicated by the j-th operation record block, and the second original dataset is obtained after the j-th updater updates the first original dataset; the initial value of j is J. If j-1 is not equal to 0, determine the position of the data blocks in the first original dataset excluding the target data block corresponding to the update data block among the M data blocks based on the j-th operation record block, and decrement j by 1. If j-1 is equal to 0, determine K data blocks from the M data blocks excluding the target data block corresponding to the update data block.

[0293] Optionally, the j-th operation record block includes a counting sequence corresponding to the updated data block and a fifth mapping sequence, wherein the verification information of the updated data block corresponds to at least one specific element in the fifth mapping sequence. The processing module 2901 can also be used to: determine the verification information of the target data block corresponding to the updated data block based on the target data block and the counting sequence corresponding to the updated data block; determine a sixth mapping sequence, wherein the verification information of the updated data block corresponds to at least one specific element in the sixth mapping sequence; and verify whether the target data block has been modified after the update is performed by the j-th updater based on the fifth mapping sequence and the sixth mapping sequence.

[0294] Optionally, the fourth dataset may also include an initial counting sequence. After determining K data blocks out of the M data blocks, the processing module 2901 may further be configured to: determine the counting sequence corresponding to each of the K data blocks based on the initial counting sequence.

[0295] Optionally, when acquiring the fourth dataset, the processing module 2901 can specifically be used to receive the fourth dataset via the transceiver module 2902.

[0296] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0297] In this embodiment, the data processing device 290 is presented in an integrated manner, divided into various functional modules. Here, "module" can refer to a specific ASIC, circuitry, a processor and memory executing one or more software or firmware programs, integrated logic circuitry, and / or other devices that can provide the aforementioned functions. In a simplified embodiment, those skilled in the art will recognize that the data processing device 290 can employ... Figure 7 The data processing device 70 shown is in the form of a data processing device.

[0298] for example, Figure 7The processor 701 in the data processing device 70 shown can execute the communication method in the above method embodiment by calling computer execution instructions stored in the memory 703.

[0299] Specifically, Figure 29 The functions / implementation process of the processing module 2901 and the transceiver module 2902 can be obtained through Figure 7 The processor 701 in the data processing device 70 shown calls computer execution instructions stored in the memory 703 to implement the function. Alternatively, Figure 29 The function / implementation process of the processing module 2901 can be obtained through Figure 7 The processor 701 in the data processing device 70 shown calls computer execution instructions stored in the memory 703 to implement the function. Figure 29 The function / implementation process of the transceiver module 2902 in the middle can be obtained through Figure 7 This is achieved through the communication interface 704 in the data processing device 70 shown.

[0300] Since the data processing device 290 provided in this embodiment can execute the above-described communication method, the technical effects it can achieve can be referred to the above-described method embodiments, and will not be repeated here.

[0301] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0302] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0303] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0304] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0305] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0306] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0307] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0308] As used herein, the terms “component,” “module,” “system,” etc., are intended to refer to a computer-related entity, which may be hardware, firmware, a combination of hardware and software, software, or running software. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, a running thread, a program, and / or a computer. As an example, an application running on a computing device and the computing device itself can both be components. One or more components may reside in a running process and / or thread, and components may be located in a single computer and / or distributed among two or more computers. Furthermore, these components are capable of execution from various computer-readable media having various data structures thereon. These components may communicate locally and / or remotely via signals, such as based on one or more data packets (e.g., data from a component that interacts with a local system, another component in a distributed system, and / or signals that interact with other systems via a network such as the Internet).

[0309] This application presents various aspects, embodiments, or features relating to systems that may include multiple devices, components, modules, etc. It should be understood and appreciated that individual systems may include additional devices, components, modules, etc., and / or may not include all the devices, components, modules, etc. discussed in conjunction with the accompanying drawings. Furthermore, combinations of these approaches are also possible.

[0310] Additionally, in the embodiments of this application, the term "exemplary" is used to indicate that it is an example, illustration, or illustration. Any embodiment or design scheme described as "exemplary" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Rather, the use of the term "exemplary" is intended to present the concept in a specific manner.

[0311] In the embodiments of this application, the terms "information," "signal," "message," and "channel" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction, their intended meanings are consistent. Similarly, "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction, their intended meanings are consistent. "System" and "network" may sometimes be used interchangeably. Without emphasizing the distinction, their intended meanings are consistent; for example, "communication network" also refers to "communication system."

[0312] The network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0313] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized by, Applied to the generating side, the method includes: Obtain N data blocks, where N is a positive integer; A first dataset is determined, which includes the N data blocks and the first mapping sequence; Wherein, the verification information of the nth data block corresponds to at least one specific element in the first mapping sequence, the first mapping sequence is used to verify whether the nth data block has been modified; the verification information of the nth data block is determined based on the nth data block and the counting sequence corresponding to the nth data block, different data blocks correspond to different counting sequences; n is a positive integer from 1 to N.

2. The method of claim 1, wherein, The method further includes: A digest of the counting sequence corresponding to the nth data block is determined based on the first key; The verification information of the nth data block is obtained by performing operations on the summary of the counting sequence corresponding to the nth data block and the nth data block.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The verification information of the nth data block is mapped to a Bloom filter through a set of hash functions to obtain an intermediate mapping sequence, wherein the set of hash functions includes at least one hash function; The first mapping sequence is determined based on the intermediate mapping sequence.

4. The method according to claim 3, characterized in that, The at least one hash function is a keyed hash function.

5. The method according to claim 3, characterized in that, Determining the first mapping sequence based on the intermediate mapping sequence includes: The intermediate mapping sequence is encrypted using the second key to obtain the first mapping sequence; Alternatively, the intermediate mapping sequence can be determined as the first mapping sequence.

6. The method according to claim 1, 2, 4, or 5, characterized in that, The method further includes: Determine a summary of the first mapping sequence; The first dataset also includes a summary of the first mapping sequence.

7. The method according to claim 1, 2, 4, or 5, characterized in that, The first dataset also includes a counting sequence corresponding to the first data block among the N data blocks; Alternatively, the first dataset may also include a counting sequence corresponding to each of the N data blocks.

8. The method according to claim 1, 2, 4, or 5, characterized in that, The method further includes: Send the first dataset.

9. A data processing method, characterized in that, Applied to the updating party, the method includes: Obtain a second dataset; the second dataset is the same as the first dataset, or the second dataset is a dataset after the first dataset has been updated at least once by other updaters; the first dataset is a dataset determined by the generator. When the second dataset is the same as the first dataset, the second dataset includes N data blocks and a first mapping sequence; When the second dataset is a dataset after the first dataset has been updated at least once by other updaters, the second dataset includes data blocks after N data blocks have been updated at least once and a first mapping sequence; Wherein, the verification information of the nth data block corresponds to at least one specific element in the first mapping sequence, the first mapping sequence is used to verify whether the nth data block has been modified; the verification information of the nth data block is determined based on the nth data block and the counting sequence corresponding to the nth data block, different data blocks correspond to different counting sequences; N is a positive integer, and n is a positive integer from 1 to N; The second dataset is updated to obtain an updated second dataset, wherein the second dataset and the updated second dataset have at least one different data block; A third dataset is determined, which includes the updated second dataset and an operation record block, the operation record block being used to instruct the updater on the second dataset.

10. The method according to claim 9, characterized in that, The update of the second dataset includes: adding a first data block to the second dataset; The operation record block includes first location information, which indicates the position of the first data block in the third dataset and the position of the update relative to the second dataset.

11. The method according to claim 10, characterized in that, The operation record block further includes a counting sequence corresponding to the first data block and a second mapping sequence. The counting sequence corresponding to the first data block is used to determine the verification information of the first data block, and the verification information of the first data block corresponds to at least one specific element in the second mapping sequence.

12. The method according to any one of claims 9-11, characterized in that, The update of the second dataset includes: modifying the second data block in the second dataset; wherein, modifying the second data block includes at least one of the following: replacing part or all of the content of the second data block, adding content to the second data block, or deleting part of the content of the second data block; The operation record block includes second location information, which indicates the position of the second data block in the third dataset and the position of the update relative to the second dataset.

13. The method according to claim 12, characterized in that, The operation record block also includes a counting sequence corresponding to the second data block and a third mapping sequence. The counting sequence corresponding to the second data block is used to determine the verification information of the second data block, and the verification information of the second data block corresponds to at least one specific element in the third mapping sequence.

14. The method according to any one of claims 9-11 or 13, characterized in that, The update of the second dataset includes: deleting the third data block in the second dataset; The operation record block includes third location information, which indicates the position of the update relative to the second dataset.

15. The method according to any one of claims 9-11 or 13, characterized in that, The operation record block further includes at least one of the following: the identifier of the updater, the time when the update occurred or ended, the type of the update, the source of the second dataset, or the signature of the operation record block; wherein the source of the second dataset is the device that generated the second dataset.

16. The method according to any one of claims 9-11 or 13, characterized in that, After determining the third dataset, the method further includes: Send the third dataset.

17. A data processing method, characterized in that, Applied to the verification side, the method includes: Obtain the fourth dataset, which includes M data blocks and a first mapping sequence, where M is a positive integer; Determine K data blocks from the M data blocks, where K is a positive integer less than or equal to M; The verification information of the kth data block is determined based on the kth data block and the corresponding counting sequence. Different data blocks correspond to different counting sequences, and k is a positive integer from 1 to K. Determine the fourth mapping sequence, wherein the verification information of the kth data block corresponds to at least one specific element in the fourth mapping sequence; Based on the first mapping sequence and the fourth mapping sequence, determine whether the kth data block has been modified.

18. The method according to claim 17, characterized in that, The fourth dataset also includes J operation record blocks, each corresponding to one of the J updaters. The operation record blocks are used to indicate the updaters that generated the operation record blocks to update the original dataset. The original dataset is the dataset before the updaters made the update. Determining K data blocks from the M data blocks includes: Based on the J operation record blocks, determine K data blocks from the M data blocks.

19. The method according to claim 18, characterized in that, The step of determining K data blocks from the M data blocks based on the J operation record blocks includes: Based on the j-th operation record block, determine the target data block corresponding to the update data block of the j-th updater among the M data blocks; wherein, the update data block is a data block in the second original dataset indicated by the j-th operation record block, and the second original dataset is obtained by the j-th updater after updating the first original dataset; the initial value of j is J; If j-1 is not equal to 0, determine the position of the data block in the first original dataset, excluding the target data block corresponding to the updated data block, among the M data blocks according to the j-th operation record block, and decrement j by 1. If j-1 equals 0, the data blocks other than the target data block corresponding to the updated data block among the M data blocks are determined as the K data blocks.

20. The method according to claim 19, characterized in that, The updated data block is the data block added or modified by the j-th updater; The j-th operation record block includes fourth location information, which indicates the position of the updated data block in the second original dataset and the position of the update performed by the updater relative to the first original dataset.

21. The method according to claim 19 or 20, characterized in that, The j-th operation record block includes a counting sequence corresponding to the updated data block and a fifth mapping sequence, and the verification information of the updated data block corresponds to at least one specific element in the fifth mapping sequence; The method further includes: Based on the target data block corresponding to the updated data block and the counting sequence corresponding to the updated data block, determine the verification information of the target data block corresponding to the updated data block; Determine the sixth mapping sequence, wherein the verification information of the updated data block corresponds to at least one specific element in the sixth mapping sequence; Verify whether the target data block has been modified after the j-th updater performs the update based on the fifth mapping sequence and the sixth mapping sequence.

22. The method according to any one of claims 17-20, characterized in that, The fourth dataset also includes a starting count sequence; After determining K data blocks out of the M data blocks, the method further includes: The counting sequence corresponding to each of the K data blocks is determined based on the initial counting sequence.

23. A data processing apparatus, characterized in that, The data processing device includes: a processor and a memory; The memory is used to store computer execution instructions, which, when executed by the processor, cause the data processing device to perform the method as described in any one of claims 1-8, 9-16, or 17-22.

24. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a computer, causes the computer to perform the method of any one of claims 1-8, 9-16, or 17-22.

Images