A trusted multi-modal data processing method and system

Abstract

The application provides a kind of trusted multi-modal data processing method and system, it is related to data processing technical field, including: with the continuity of the accurate matching of the content fingerprint of the slice and the continuity of the flow track as the classification boundary, the plurality of candidate data slices are classified, and the plurality of candidate data slices are divided into effective slice set and abnormal slice set;For each effective data slice in the effective slice set, along the spatial splicing track of the effective data slice to the reconstruction center, the time sequence accumulation decoding is carried out slice by slice, to obtain the recovered data by synthesis in the reconstruction center;The reconstruction content fingerprint of the recovered data is calculated, and the reconstruction content fingerprint is compared with the original content fingerprint, and the evidence recovery verification result of multi-modal data is output.The application can effectively prevent historical evidence from being tampered with by a single point, and realize the traceability of uploading behavior and business action.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a reliable multimodal data processing method and system. Background Technology

[0002] In the digital environment, electronic data such as videos, images, documents, contracts, and records have become important carriers of personal expression, business operations, government administration, and judicial activities. However, most existing electronic data is stored on centralized platforms or single business systems, which have long suffered from problems such as susceptibility to tampering and deletion, difficulty in determining liability, incomplete chains of evidence, and difficulty in recovery after failures.

[0003] While traditional centralized storage facilitates unified management, it inherently carries the risk of a single point of failure. In the event of server failure, accidental deletion, malicious attacks, or unauthorized internal modifications, the authenticity and integrity of electronic data may be questioned. Especially in scenarios involving copyright protection, process tracking, business credentials, and judicial evidence collection, simply relying on backend logs or ordinary backups often fails to provide sufficiently credible technical proof.

[0004] In existing technologies, some solutions generate digests of uploaded data using cryptographic hash functions and record these digests in a blockchain or chained log to ensure the non-repudiation of the upload. However, such solutions typically only focus on the preservation of content at the moment of upload, failing to address how to recover lost content and whether the recovered results remain reliable. Specifically, existing technologies often design immutability, recoverability, and traceability as separate capabilities: some solutions emphasize hash verification but cannot recover the original data; some emphasize backup and recovery but cannot prove that the recovered result matches the original uploaded content; and some solutions can retain operation logs but lack a unified and trusted verification chain.

[0005] On the other hand, in distributed storage scenarios, data is split and stored across multiple logical nodes. When some nodes fail or data fragments are tampered with during transmission, existing recovery solutions typically only collect and piece together the fragments based on their sequence number or storage location. They lack the ability to track the actual flow of data fragments in the distributed network and also lack a dual verification mechanism for the integrity of fragment content and the continuity of the flow path. This leads to situations where, in the event of node anomalies or network attacks, tampered fragments may be mistakenly considered valid fragments and participate in reconstruction, causing the recovery result to deviate from the original content, and the system cannot identify this deviation.

[0006] Furthermore, existing solutions, when generating data fingerprints, often only perform single-dimensional hash calculations on the original byte stream, without considering the structural relationships of the data content in a multi-dimensional feature space. When the data content undergoes minor local tampering, although a single-dimensional hash fingerprint can theoretically detect changes, in practical engineering scenarios, if the recovery process introduces problems such as fragmentation misalignment, redundant check failures, or splicing offsets, relying solely on byte-stream-level fingerprint comparison is insufficient to accurately pinpoint the source of recovery deviations and cannot provide structured and reliable evidence for the auditing system. Summary of the Invention

[0007] This invention provides a trusted multimodal data processing method that can effectively prevent historical evidence from being tampered with at a single point and enable traceability of upload behavior and business actions.

[0008] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: Firstly, a reliable multimodal data processing method includes: Step 1: Obtain the original content fingerprint of the multimodal data; along the temporal flow path of the operation behavior in the trusted ledger, perform chain hash calculation on the original content fingerprint, the current operation subject identifier, the current operation timestamp, and the hash digest of the previous evidence node to generate the current operation evidence node, so as to form a continuous chain of operation evidence. Step 2: When the data recovery command is triggered, track the spatial distribution and flow trajectory of multiple candidate data fragments in the distributed storage network to obtain multiple candidate data fragments; use the accurate matching of fragment content fingerprints and the continuity of flow trajectory as classification boundaries to verify and classify multiple candidate data fragments, and divide multiple candidate data fragments into a set of valid fragments and a set of abnormal fragments. Step 3: For each valid data fragment in the valid fragment set, perform temporal cumulative decoding along the spatial splicing trajectory of the valid data fragments converging towards the reconstruction center, so as to synthesize the recovered data at the reconstruction center; calculate the reconstruction content fingerprint of the recovered data, compare the reconstruction content fingerprint with the original content fingerprint, and output the evidence recovery verification result of the multimodal data.

[0009] Secondly, a trusted multimodal data processing system includes: The acquisition module is used to acquire the original content fingerprint of multimodal data; along the temporal flow path of the operation behavior in the trusted ledger, the original content fingerprint, the current operation subject identifier, the current operation timestamp, and the hash digest of the previous evidence node are chained hashed to generate the current operation evidence node, so as to form a continuous operation evidence chain. The judgment module is used to track the spatial distribution and flow trajectory of multiple candidate data fragments in the distributed storage network when a data recovery command is triggered, and obtain multiple candidate data fragments; using the accurate matching of fragment content fingerprints and the continuity of flow trajectory as classification boundaries, the multiple candidate data fragments are verified and classified into a set of valid fragments and a set of abnormal fragments. The calculation module is used to perform temporal cumulative decoding on each valid data fragment in the valid fragment set along the spatial splicing trajectory of the valid data fragments converging towards the reconstruction center, so as to synthesize the recovered data at the reconstruction center; calculate the reconstruction content fingerprint of the recovered data, compare the reconstruction content fingerprint with the original content fingerprint, and output the evidence recovery verification result of the multimodal data.

[0010] The above-described solution of the present invention has at least the following beneficial effects: By forming a continuous chain of operational evidence, the temporal flow path of the operational behavior has an irreversible unidirectional temporal constraint on the hash association between the preceding and following evidence nodes. In non-chained evidence storage scenarios, this constraint cannot be reproduced due to the lack of dependencies between preceding and following nodes. Therefore, this mechanism can effectively prevent historical evidence from being tampered with at a single point, and achieve traceability and non-repudiation of upload behavior and business actions.

[0011] When a data recovery command is triggered, the spatial distribution and flow trajectory of multiple candidate data fragments in the distributed storage network are tracked, rather than simply collected based on fragment number or storage location. Because the spatial distribution and flow trajectory of candidate data fragments impose strict path fidelity constraints on the validity of data fragments—constraints that do not exist in scenarios where data is directly distributed from the central node due to the lack of a distributed flow process—this mechanism can accurately reconstruct the actual transmission status of each fragment in a distributed network environment.

[0012] Candidate data fragments are divided into a set of valid fragments and a set of abnormal fragments. Since the precise matching of fragment content fingerprints can detect whether the fragment content has been tampered with, and the continuity of the flow trajectory can detect whether the fragment has been abnormally interrupted or broken during transmission, the classification boundary formed by the combination of the two does not exist in the fuzzy verification scenario due to the lack of content consistency requirements and path integrity requirements. Therefore, this mechanism can simultaneously exclude two types of risks: content tampering and transmission abnormalities, and avoid abnormal fragments from polluting the subsequent reconstruction process.

[0013] The data is synthesized by performing temporal cumulative decoding on a piecewise basis. Since the spatial splicing trajectory of the effective data pieces converging to the reconstruction center has an exclusive content fidelity constraint on the reconstruction of the complete data, this constraint becomes invalid in the global reconstruction scenario that includes abnormal pieces due to the intervention of abnormal pieces. Therefore, this mechanism can ensure that only the effective pieces that have undergone double verification are used to participate in the reconstruction, thus guaranteeing the content integrity of the recovered data from the source. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a trusted multimodal data processing method according to an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of a trusted multimodal data processing system according to an embodiment of the present invention. Detailed Implementation

[0016] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0017] This invention provides a trusted multimodal data processing method and system, which is designed for multimodal electronic data such as videos, pictures, documents, contracts, and records. It establishes a unified data content fingerprint and identity binding mechanism to form an immutable chain of operational evidence. When data is missing or damaged, it enables trusted recovery and verification of the original data through a distributed storage network.

[0018] like Figure 1 As shown, this embodiment uses video data as a typical carrier to describe the technical solution of the present invention in detail. The technical solution of the present invention is not limited to video data, but is also applicable to multimodal electronic data such as images, text, electronic contracts, medical records, and test reports. In this embodiment, the trusted ledger is implemented using a chained hash log structure, and the storage nodes in the distributed storage network are multiple physically dispersed logical storage units interconnected through the network. The cryptographic digest rule uses the SHA-256 secure hash algorithm, and the erasure coding uses the Reed-Solomon encoding scheme.

[0019] Step 1: Obtain the original content fingerprint of the multimodal data and construct the operational evidence chain. The original content fingerprints of multimodal data are obtained and a continuous chain of operational evidence is formed by following the temporal flow path of the operation behavior in the trusted ledger. The following example, the process of uploading and storing a video file, illustrates the specific implementation of step 1 in detail.

[0020] When a user submits a video file through the client, the system first reads the raw byte stream of the video file. This raw byte stream is loaded from the file storage medium into a memory buffer in binary form. Simultaneously, the system obtains the identity information of the current operator. This identity information can come from the user's login account, digital certificate, device hardware identifier, or a combination thereof. In this embodiment, the unique identifier of the user account is used as the current operator's identifier. Furthermore, the system obtains the current precise time from a network time protocol service or a clock and records it as the current operation timestamp.

[0021] Feature extraction is performed on the original byte stream of the video file according to a pre-defined cryptographic digest rule. Specifically, all bytes of the video file are fed into the SHA-256 hash function in their original order to calculate a fixed-length digest value of 256 bits (32 bytes), denoted as the original data digest. This process can be described as follows: starting from the first byte of the video file, each byte value is read sequentially, and all byte values ​​are input into the compression iteration operation of the secure hash algorithm. After multiple rounds of message expansion and compression, the digest is output. The one-way nature of the SHA-256 algorithm ensures the feasibility of calculating a digest from the original data, while its collision resistance ensures that the probability of different data producing the same digest is computationally infeasible, thus enabling the original data digest to uniquely represent the original content of the video file.

[0022] After obtaining the original data digest, it undergoes collision resistance checking and length normalization. Collision resistance checking refers to checking whether the values ​​of the original data digest fall within the valid range of the SHA-256 digest space, that is, whether its binary length is 256 bits and whether each bit is a valid value. Length normalization refers to converting the 256-bit original data digest into a fixed-length hexadecimal string of 64 characters according to a preset encoding rule (hexadecimal lowercase encoding is used in this embodiment). This fixed-length string is the original content fingerprint. The original content fingerprint has the following characteristics: if any byte of the original data changes, the corresponding original content fingerprint will undergo unpredictable character changes at different positions, thus allowing the detection of data tampering by comparing the original content fingerprint. The original content fingerprint of this video file can be recorded as follows: .

[0023] Construct an operational evidence chain, which resides in a trusted ledger with a chain structure. Each node in this trusted ledger (i.e., an evidence node) records irrefutable evidence of a particular operational action. The ledger is structured as a singly linked list, with each node containing its hash digest and a reference to the previous node (implemented through the hash digest of the previous node).

[0024] The current operation evidence node is generated according to the following process: Based on the original content fingerprint, the current operation subject identifier, the current operation timestamp, and the hash digest of the previous evidence node, the fields are combined according to a preset temporal concatenation rule to obtain the current operation evidence element set. The temporal concatenation rule defines the order in which the fields are concatenated: the original content fingerprint (64-character hexadecimal string) is placed first, followed by the current operation subject identifier (variable-length string, ending with a null character), then the current operation timestamp (fixed-format 24-character UTC time string), and finally the hash digest of the previous evidence node (64-character hexadecimal string). If the current operation is the first operation in this ledger (i.e., there is no previous evidence node), then the hash digest field of the previous evidence node uses a 64-character hexadecimal string of all zeros (i.e., 32 bytes of zero value) as the genesis node identifier. These four parts of data are concatenated end-to-end in the above order to obtain the current operation evidence element set.

[0025] Based on the current set of operational evidence elements, secondary feature extraction is performed according to a preset cryptographic digest rule (also using the SHA-256 algorithm). The concatenated complete byte sequence is used as input to the SHA-256 algorithm to calculate a new 256-bit fixed-length digest. This digest value is the hash digest of the current operational evidence node. This current operational evidence node (denoted as...) It not only records information about the current operation itself (original content fingerprint, operation subject, operation time), but also establishes an immutable association with historical operation records by including the hash digest of the previous node.

[0026] Based on the current operation evidence node, it is appended to the end node of the trusted ledger. Specifically, the trusted ledger maintains an ordered list of evidence nodes in memory. When a new current operation evidence node is generated, its complete data structure, including the original content fingerprint, operation subject identifier, operation timestamp, current node hash digest, and predecessor reference pointing to the previous node, is appended to this list as a new record. Simultaneously, the updated ledger state is persistently written to disk storage to ensure that ledger data is not lost after a restart. This results in a continuous chain of operation evidence, where the first... Nodes With the Nodes An irreversible one-way temporal constraint is formed between them through hash references: any interaction with historical nodes... Any alteration to any field in the system will result in The hash value changes, thus causing... Due to changes in references during generation Different hash values ​​are obtained, thus rendering the entire chain of evidence invalid from the node that was tampered with.

[0027] To further illustrate the chained hash calculation process of the operational evidence chain, a simplified numerical example is provided below. Assume that the concatenated byte sequence obtained after encoding the four fields of the current operational evidence element set is... The byte sequence obtained after decoding the hash digest of the previous evidence node is: Then the hash value of the current operation evidence node .because It already contains The encoded value, so actually The calculation implicitly depends on the hash value of the previous node. When a user uploads a second file, a second evidence node is generated. The hash value depends on The hash value; when uploading a third file, The hash value depends on The hash value; and so on. This chain dependency ensures that the auditor only needs to verify the correctness of the hash value node by node backward from the latest evidence node to confirm that the entire evidence chain has not been tampered with since its generation.

[0028] After constructing the operational evidence chain, the original multimodal data needs to be distributed and stored. In this embodiment, erasure coding fragmentation is used to encode and fragment the original video file. Specifically, the original byte stream of the video file is divided into blocks of a fixed size of 64KB (i.e., 65536 bytes). Each data block is further organized into a logical matrix and generated using the Reed-Solomon erasure coding algorithm. One verification block (taken in this embodiment) , That is, 10 data blocks generate 4 redundant check blocks, for a total of 14 fragments. Each fragment is assigned a unique fragment number (from 0 to 13), and a fragment content fingerprint is calculated for each fragment (by performing a SHA-256 operation on the byte stream of the fragment to obtain the corresponding fragment content fingerprint). These 14 fragments and their corresponding fragment content fingerprints are sent to 14 different storage nodes in the distributed storage network for storage. At the same time, a fragment location mapping table is maintained in the central index, recording the storage node identifier corresponding to each fragment number and the offset position of that fragment in the original data.

[0029] Step 2: Data Recovery Process and Fragment Verification and Classification When original data is lost or damaged due to storage node failure, network transmission errors, or accidental deletion, a data recovery command is triggered, and the data recovery process begins. The following details the specific implementation process of step 2.

[0030] Based on the data recovery command, the target data identifier is extracted. The data recovery command can be initiated manually by the administrator or automatically triggered when a data integrity verification failure is detected. The command carries a unique identifier for the target data to be recovered. This identifier is assigned when the data is first uploaded for evidence storage (in this embodiment, a UUID format data identifier is used). The target data identifier is extracted from the recovery command.

[0031] Based on the target data identifier, the node routing table in the distributed storage network is queried to obtain multiple candidate storage node identifiers corresponding to the target data. The node routing table is a globally maintained mapping data table in the distributed storage network, recording the correspondence between each data identifier and the node of its storage shard. The data structure of the node routing table is a set of key-value pairs, where the key is the target data identifier and the value is a list of node identifiers. Each item in the list contains the storage node identifier and the shard number stored on that node. Using the target data identifier as the query key, an exact match is performed in the node routing table to obtain the list of candidate storage node identifiers corresponding to the target data. For example, the search results return: node N_003 stores shards 0 and shard 1, node N_007 stores shard 2, node N_012 stores shards 3 and shard 4, and so on up to node N_042 storing shards 12 and shard 13. These node identifiers constitute the set of candidate storage node identifiers.

[0032] After obtaining the candidate storage node identifiers, a fragment retrieval request is sent to the corresponding candidate storage node based on the multiple candidate storage node identifiers. Specifically, a fragment retrieval request message is sent to each candidate storage node via a network communication protocol (in this embodiment, the gRPC protocol based on TCP is used). The payload of this request message includes the target data identifier and a list of fragment sequence numbers that need to be returned by the node. After receiving the request, the candidate storage node retrieves the corresponding fragment data (including fragment payload data and fragment metadata) from its local storage and encapsulates it into a response message and returns it to the requester. After receiving the response messages from each node, the fragment payload data, fragment sequence number, fragment content fingerprint, and storage node location information of each fragment are extracted, thereby receiving and retrieving multiple candidate data fragments.

[0033] While acquiring candidate data fragments, it is also necessary to acquire the spatial distribution status and flow trajectory of each candidate data fragment. The spatial distribution status refers to the location information of the storage node currently hosting each candidate data fragment within the network topology, including the node's IP address, network region (such as availability zone identifier), and node hardware type (such as SSD storage node or HDD storage node). The flow trajectory refers to the record of all transmission path nodes traversed by each candidate data fragment in the distributed storage network from its initial distribution to the storage node until the current moment. This flow trajectory consists of a series of path node entries, each entry recording the node identifier of an intermediate node, the timestamp of the node receiving the fragment, and the timestamp of the node forwarding the fragment. This information comes from the following two aspects: The first is the transmission log recorded by the distribution scheduling module during the initial distribution phase of the shard; the second is the transmission log recorded by the participants (source node and target node) in the transfer process when the shard is transferred between nodes due to node failure, load balancing or data migration.

[0034] This information is extracted from the transmission logs of each node to obtain the spatial distribution and flow trajectory of multiple candidate data fragments.

[0035] After obtaining candidate data fragments and their spatial distribution and flow trajectory, the candidate data fragments are verified and classified to distinguish between valid and abnormal fragments. The verification and classification are based on two parallel conditions: exact matching of fragment content fingerprints and continuity of the flow trajectory. Candidate data fragments that satisfy both conditions are marked as valid fragments, while candidate data fragments that do not satisfy either condition are marked as abnormal fragments.

[0036] The specific execution process for classifying and verifying data is as follows: Fragment Content Fingerprint Matching and Verification: Based on the multiple candidate data fragments, feature extraction is performed according to preset cryptographic digest rules to obtain fragment content fingerprints corresponding to the multiple candidate data fragments. This step is consistent with the generation method of the original content fingerprint: SHA-256 hash operation is performed on the fragment payload data (i.e., the byte fragments of the original data contained in the fragment) of each candidate data fragment to obtain a 256-bit fixed-length digest value corresponding to the fragment, which is then encoded in hexadecimal to form the current fragment content fingerprint of the candidate data fragment. For example, for the candidate data fragment with fragment number 3, assuming its fragment payload data is a 64KB byte sequence, the byte sequence is read byte by byte from beginning to end and input into the SHA-256 compression function to calculate the current fragment content fingerprint value corresponding to the fragment. .

[0037] Based on the current fragment content fingerprint, a precise match is performed with the original fragment fingerprint database corresponding to the target data. The original fragment fingerprint database is established when the data is first uploaded for evidence storage and fragmentation is completed, and is stored in a central index. This fingerprint database records each fragment sequence number and its corresponding original fragment content fingerprint (i.e., the SHA-256 value when the fragment is first generated). The current fragment content fingerprint of each candidate data fragment is retrieved one by one, and the original fingerprint entry with the same fragment sequence number is searched in the original fragment fingerprint database for character-by-character comparison. If the current fingerprint is completely identical to the original fingerprint in 64 hexadecimal characters (i.e., the 256 bits of SHA-256 are completely identical), the candidate data fragment is determined to have passed the content integrity check, and the fragment content fingerprint matching result is marked as a match; if any character is different, it indicates that the fragment has undergone content changes (tampering or damage) during storage or transmission, and the fragment content fingerprint matching result is marked as a mismatch.

[0038] Transmission trajectory continuity verification: Based on the transmission trajectory, extract the timestamps of adjacent path nodes during the transmission process for each candidate data fragment, calculate the time interval between adjacent path nodes, and obtain the continuity verification result of the transmission trajectory. The specific implementation details of this step are as follows: Based on the flow trajectory, the identifiers of each path node traversed by each candidate data fragment during transmission are extracted, resulting in a path node identifier sequence. Taking candidate data fragment number 5 as an example, suppose its flow trajectory shows that this fragment first starts from the distribution scheduling node during initial distribution. Send, via routing nodes Transit point, ultimately reaching the storage node. Storage; subsequently due to storage nodes If an overload warning is detected, the shard will be automatically migrated to a storage node. Passing through routing nodes Transit point. The extracted path node identifier sequence is: .

[0039] Based on the path node identifier sequence, the transmission logs of the corresponding nodes are queried to extract the data reception timestamps and data transmission timestamps for each path node. The transmission log is a structured log file maintained locally by each node participating in fragment transmission. Each log record contains: fragment identifier, transmission direction (receive or send), timestamp, and peer node identifier. Using the fragment sequence number as an index, the transmission logs of each node in the path node identifier sequence are queried to extract the receiving timestamp of that node for that fragment. ) and sending timestamp ( For example, for a node Extract the received timestamp and sending timestamp For nodes Extract the received timestamp And so on. These timestamps together form the set of timestamps for adjacent path nodes.

[0040] Based on the set of timestamps of adjacent path nodes, calculate the time difference between the data transmission timestamp of the previous node and the data reception timestamp of the next node in the adjacent path nodes. Specifically, for each pair of adjacent nodes in the path node identifier sequence... Calculate the time difference For example, nodes Send to node The time difference of reception is: Perform this calculation on all adjacent node pairs to obtain the set of time intervals between adjacent path nodes.

[0041] Based on the set of time intervals, each time interval is compared with a preset normal transmission delay threshold. The preset normal transmission delay threshold is determined based on the network topology and link characteristics of the distributed storage network: for transmission between nodes within the same region (e.g., within the same availability zone), the normal delay threshold is set to 50 milliseconds; for transmission between nodes across regions (e.g., between different availability zones), the normal delay threshold is set to 200 milliseconds. For each time interval... Obtain the network region information of the two corresponding nodes, and select an appropriate threshold based on the region relationship; then... Compare with this threshold: If If the delay is less than or equal to the applicable threshold, the delay determination result corresponding to that time interval is normal; if If the delay exceeds the applicable threshold, the result of the delay determination is considered abnormal.

[0042] Based on the delay determination results, the proportion of time intervals with a normal delay determination result is calculated out of the total number of time intervals. Let there be... There are 3 pairs of adjacent nodes, among which If the delay judgment result is normal, then the continuity ratio .when When the continuity threshold is greater than or equal to a preset continuity threshold (0.8 in this embodiment, meaning that more than 80% of the transmission segments have normal delay), the flow trajectory of the candidate data fragment is determined to be continuous, and the continuity verification result of the flow trajectory is continuous; when If the delay is less than a preset continuity threshold, it is considered discontinuous. The basis for setting this threshold is that: a small number of transmission segments experiencing delays exceeding the normal range may be caused by transient network congestion and does not necessarily indicate that the fragment was maliciously intercepted or tampered with during transmission; however, if a large number of transmission segments experience abnormal delays, it indicates that there is a problem with the fragment's transfer path and it is not suitable as a reliable source of recovery.

[0043] The classification decision is based on the fingerprint matching results of the fragment content and the continuity verification results of the flow trajectory. The classification is performed using the precise matching of the fragment content fingerprint and the continuity of the flow trajectory as the classification boundaries. Specifically: for each candidate data fragment, if its fragment content fingerprint matching result is a match (i.e., the current fingerprint is completely consistent with the original fingerprint) and the continuity verification result of the flow trajectory is continuous, then the classification label assigned to the candidate data fragment is valid; otherwise, the classification label assignment is abnormal.

[0044] Based on the classification labels, multiple candidate data fragments are divided into a valid fragment set and an abnormal fragment set. Specifically, two empty data containers are created: a valid fragment list and an abnormal fragment list. Then, all candidate data fragments are traversed, and the metadata (fragment number, fragment payload data, fragment content fingerprint) of fragments with the classification label "valid" are placed into the valid fragment set; fragments with the classification label "abnormal" are placed into the abnormal fragment set. Fragments in the abnormal fragment set do not participate in the subsequent data reconstruction process, but their information (fragment number, reason for abnormality, content fingerprint mismatch, or discontinuous flow trajectory) will be recorded in the audit log for administrators to troubleshoot. Through the above dual verification mechanism, fragments whose content has been tampered with (fingerprint mismatch) and fragments with abnormal path behavior during transmission (discontinuous trajectory) are excluded, ensuring that all fragments participating in the subsequent reconstruction are rigorously verified and trustworthy fragments.

[0045] Step 3: Data reconstruction and evidence recovery verification For each valid data fragment in the obtained valid fragment set, temporal cumulative decoding is performed along the spatial splicing trajectory converging from the valid data fragment to the reconstruction center, so as to synthesize the recovered data at the reconstruction center; then, the reconstructed content fingerprint of the recovered data is calculated and compared with the original content fingerprint, and the evidence recovery verification result is output. The specific implementation process of each sub-step is described in detail below.

[0046] Based on the set of valid fragments, the fragment sequence number and fragment payload data corresponding to each valid data fragment are extracted to obtain the fragment metadata set. Each item in the fragment metadata set is a triple (sequence number, payload data, content fingerprint), which records the number of the valid fragment, the actual data bytes, and the verification fingerprint.

[0047] Based on the fragment sequence number in the fragment metadata set, a preset fragment location mapping table is queried to obtain the target offset position corresponding to each valid data fragment. The fragment location mapping table is a static mapping data table established when data is first fragmented and stored. This table records the correspondence between each fragment sequence number and its byte offset range in the original complete data. For example, the record in the fragment location mapping table is: Fragment sequence number 0 corresponds to the target offset position... Segment number 1 corresponds to the target offset position. And so on, segment number Corresponding target offset position Using the fragment number as the key, a lookup operation is performed in the fragment location mapping table to obtain the target offset position corresponding to that fragment, which is the start and end byte offset of that fragment in the recovered complete data. After obtaining the target offset position of each valid fragment, the valid data fragments are sorted in ascending order of target offset position to obtain an ordered fragment queue. During the sorting process, the fragment metadata set is rearranged using the start offset of the target offset position of each fragment as the sort key. After sorting, the fragments in the ordered fragment queue are arranged according to the order in which they appeared in the original complete data.

[0048] According to the ordered fragment queue, the fragment payload data of each valid data fragment is extracted sequentially and concatenated according to the corresponding target offset position. Specifically, a contiguous memory buffer is allocated in memory as a reconstruction buffer, the total size of which is equal to the total number of bytes of the original complete data. Starting from the head of the ordered fragment queue, the fragment payload data of each fragment is extracted sequentially, and the byte sequence of the fragment is copied byte by byte to the corresponding target offset position in the reconstruction buffer. For example, for an offset position of... The data is divided into segments, and the 65536 bytes of payload data in each segment are copied to positions 65536 to 131071 of the reconstruction buffer. This process is performed segment by segment, with the payload data of each segment placed at its correct offset. After all valid segments have been processed, the contents of the reconstruction buffer constitute the initial reconstruction data stream, the length of which is equal to the total number of bytes of the original complete data, with the content at each byte position derived from the data at that position in the payload data of the corresponding segment.

[0049] Since not all original fragments are necessarily recycled as valid fragments (some fragments may be classified as abnormal fragments due to content fingerprint mismatch or discontinuous flow trajectory), the initial reconstructed data stream may contain missing byte segments. Therefore, based on the initial reconstructed data stream, data error correction and integrity restoration are performed according to preset erasure coding decoding rules. In this embodiment, the erasure coding adopts the Reed-Solomon encoding scheme, which tolerates a maximum number of missing redundant check blocks. (i.e., the number of verification blocks). Specifically, the initial reconstructed data stream corresponding to... Each fragment ( Data shards and Each parity shard is considered a Reed-Solomon codeword. For shards present in the valid shard set, their data is known; for missing shards (belonging to the abnormal shard set or failing to be reclaimed from the storage node), their data is unknown and needs to be recovered by solving a system of linear equations using existing shards (whether data shards or parity shards). As long as the number of existing shards is not less than... If a condition is met (i.e., the minimum recovery threshold of Reed-Solomon encoding is reached), then matrix inversion can be used to utilize existing... A linear combination of the fragments uniquely recovers all the The original data is divided into fragments. After erasure coding decoding, all byte positions in the reconstruction buffer are correctly filled, resulting in the recovered data. This recovered data is the multimodal data that is consistent with the original uploaded content, obtained by assembling and correcting the valid fragments after double verification.

[0050] After obtaining the recovered data, the reconstructed content fingerprint of the recovered data is calculated and compared with the original content fingerprint obtained in step 1 to determine whether the recovery result is reliable. This process not only includes traditional byte-stream level cryptographic fingerprints, but also introduces spatial geometric structure features based on multi-dimensional feature space to form a joint fingerprint, thereby enhancing the ability to detect recovery deviations.

[0051] Feature point mapping: Based on the recovered data, it is divided into multiple data sub-blocks according to a preset dimensional mapping rule, and the features of each data sub-block are mapped to a multi-dimensional feature space to obtain multiple data feature points. Specifically, the complete byte stream of the recovered data is divided into equal-length sub-blocks with a fixed step size (in this embodiment, each sub-block is 4096 bytes) to obtain... Sub-block For each data sub-block Extract its multidimensional features: Feature dimensions include the byte mean of the sub-block (arithmetic mean of all byte values), byte standard deviation (square root of the square mean of the deviations of each byte value from the mean), byte entropy (information entropy based on the frequency distribution of byte values, reflecting the randomness of the sub-block content), and the header jump value of the sub-block (the value before the header jump value of the sub-block). (The difference between the maximum and minimum values ​​of each byte). For each sub-block Its four eigenvalues ​​constitute a four-dimensional coordinate vector. This coordinate vector is the data sub-block. A data feature point in a multidimensional feature space. All Each block The data feature points constitute the data feature point set.

[0052] Constructing a topology graph: Based on the multiple data feature points, calculate the spatial geometric distance between any two data feature points, and construct a spatial geometric topology graph based on the spatial geometric distance. The specific implementation of this process is as follows: Traverse all data feature point pairs in the data feature point set Extract the four-dimensional coordinates of each pair of data feature points. and Calculate the Euclidean distance between the two. The Euclidean distances between all pairwise data feature points are summed to obtain the spatial geometric distance set. .

[0053] According to the set of spatial geometric distances Two data feature points with a distance less than a preset threshold are marked as adjacent nodes, generating node connection relationships. The preset distance threshold is set based on the empirical distribution of data sub-block features: in this embodiment, a distance set is used. The distance threshold is 0.3 times the median, denoted as . The threshold is set based on the principle that connected adjacent nodes should have sufficiently similar content features, while non-connected nodes should exhibit distinguishable differences in their content features. For any two data feature points... and ,like Then in and Establish an undirected edge between them. All The data feature points and the edges between them together constitute a spatial geometric topology graph. , where the vertex set edge set .

[0054] Adjacency matrix and eigenvalue calculation: Based on the node connection relationships in the spatial geometric topology graph, a geometric adjacency matrix is ​​constructed; all node identifiers in the spatial geometric topology graph are extracted and sorted in ascending order according to a preset identifier sorting rule to obtain an ordered node identifier sequence. The total number of node identifiers is counted to obtain the matrix dimension parameters. Based on the matrix dimension parameter Generate a dimension of size The all-zero matrix is ​​used as the initial square matrix; based on the edge set Each edge in In the initial square matrix, the first Line number The column element values ​​are updated to the preset connection identifier value 1 (representing a node). With nodes There is an edge between them); at the same time, the first Line number The column elements are also updated to 1 (because the edges are undirected); after traversing all edges, the geometric adjacency matrix is ​​obtained. .

[0055] According to the geometric adjacency matrix Calculate its eigenvalues; construct the matrix. ,in for The identity matrix, The variable is unknown. Calculate the determinant of this matrix. The determinant is a set of... of The polynomial of degree, i.e., the target characteristic polynomial For larger matrices, the QR iterative algorithm is used: based on the geometric adjacency matrix... For input, Perform multiple rounds of QR decomposition and recombination iterations ( , ), until The matrix converges to an upper triangular matrix, meaning the elements on the main diagonal converge to the eigenvalues. Extracting eigenvalues ​​from the main diagonal of the converged matrix... Each value is used as a candidate feature root.

[0056] Residual verification and invariant calculation: To ensure the accuracy of candidate eigenvalues, each candidate eigenvalue is substituted into the target characteristic polynomial to perform residual calculation. That is, for each candidate eigenvalue... Calculate the residual value If the residual value Less than the preset residual threshold (in this embodiment, it is taken as...) ), then it is believed If a feature is valid, it is added to the feature set; otherwise, it is discarded. The final feature set is obtained. ,in .

[0057] Obtain the set of eigenvalues Then, the sum of squares of each eigenvalue is calculated to obtain the first geometric invariant. ,Right now This invariant reflects the total energy of connections between nodes in a spatial geometric topology graph, and in algebraic graph theory, it is associated with the total number of closed paths of length 2 in the graph. Simultaneously, by calculating the sum of the cubes of each eigenvalue, the second geometric invariant is obtained. ,Right now This invariant is associated with the total number of closed paths (triangular paths) of length 3 in the graph. The two geometric invariants capture global structural information of the spatial geometric topology graph from different orders: It is mainly affected by the edge density of the topological graph (i.e., the density of connections between nodes), and It is mainly affected by the triangular substructure in the topological graph (i.e., the local clustering pattern in which three nodes are connected in pairs).

[0058] Spatial geometric structure feature generation: based on the first geometric invariant and the second geometric invariant The spatial geometric structure features are obtained by combining them according to preset field concatenation rules. The field concatenation rules are as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] The IEEE 754 binary representation of a 64-bit double-precision floating-point number and The IEEE 754 binary representation of a 64-bit double-precision floating-point number is concatenated end-to-end to form a 128-bit binary sequence, which represents the spatial geometric structure feature. .

[0059] In obtaining spatial geometric structural features Next, the joint feature extraction stage begins. This is based on the original byte stream of the recovered data and the spatial geometric features. Joint feature extraction is performed according to preset cryptographic digest rules. Specifically, the complete original byte stream of the data (length is...) is recovered. (bytes) and spatial geometric features (16 bytes in length) concatenated into a single string of length. A combined data block of bytes is input into the SHA-256 hash function to calculate a new 256-bit fixed-length digest. This digest, after hexadecimal encoding, becomes the fingerprint of the reconstructed content. The reconstructed content fingerprint simultaneously encodes information from two dimensions of the recovered data: first, the complete content at the byte stream level (reflected through the original byte stream participating in hash calculations), and second, the structural relationships in the multi-dimensional feature space (reflected through spatial geometric structural features participating in hash calculations).

[0060] Based on the reconstructed content fingerprint and the original content fingerprint Then, perform a character-level comparison. Character-level comparison refers to comparing two 64-character hexadecimal strings... and Starting from the first character, the fingerprints are compared one by one at corresponding positions to see if they are the same. The positions of the characters that differ during the comparison process and the content of the differences are recorded, and the summaries are used to obtain the fingerprint comparison difference value. For example, if two fingerprints are completely identical, the fingerprint comparison difference value is zero (the number of differing characters is zero); if the 10th character is different, The 10th character and If the 10th character is different, then the fingerprint comparison difference value contains at least one difference record at that position.

[0061] Based on the fingerprint comparison difference value, it is determined whether the fingerprint comparison difference value is zero. If the fingerprint comparison difference value is zero (i.e., the 64 characters of the two fingerprints are exactly the same), the consistency judgment result is consistent, indicating that the recovered data is completely consistent with the original data at both the byte level and the spatial structure level, and no deviation was introduced during the recovery process. If the fingerprint comparison difference value is not zero (i.e., the two fingerprints differ in one or more characters), the consistency judgment result is inconsistent, indicating that there is a deviation between the recovered data and the original data at the byte level or the spatial structure level. Regardless of whether the deviation is caused by fragment misalignment, parity block decoding error, or data corruption during transmission, it is detected.

[0062] Based on the consistency judgment result, a corresponding data recovery trusted status identifier is generated, and the evidence recovery verification result is obtained. Specifically: If the consistency assessment result is consistent, the data recovery trust status is marked as trustworthy recovery. The evidence-based recovery verification result includes: target data identifier, recovery timestamp, consistency assessment result, reconstructed content fingerprint value, original content fingerprint value, number of valid fragments participating in the recovery, number of abnormal fragments, and a summary of the reasons for the abnormalities. This evidence-based recovery verification result can serve as a technical proof of the authenticity and integrity of the recovered data, and can be used for subsequent auditing, rights protection, or judicial evidence.

[0063] If the consistency assessment result is inconsistent, the data recovery trust status is marked as untrusted recovery. In addition to the basic information mentioned above, the evidence recovery verification result also includes the location and content of the differences recorded in the fingerprint comparison difference values, so that the administrator can locate the source of the recovery deviation. This untrusted recovery result is marked as an untrusted temporary result and will not replace the original trusted credentials; it is only used as a reference for risk warnings and problem analysis.

[0064] like Figure 2 As shown, a trusted multimodal data processing system includes: The acquisition module is used to acquire the original content fingerprint of multimodal data; along the temporal flow path of the operation behavior in the trusted ledger, the original content fingerprint, the current operation subject identifier, the current operation timestamp, and the hash digest of the previous evidence node are chained hashed to generate the current operation evidence node, so as to form a continuous operation evidence chain. The judgment module is used to track the spatial distribution status and flow trajectory of multiple candidate data fragments in the distributed storage network when a data recovery command is triggered, and obtain multiple candidate data fragments; using the accurate matching of fragment content fingerprints and the continuity of flow trajectory as classification boundaries, the multiple candidate data fragments are verified and classified, and the multiple candidate data fragments are divided into a set of valid fragments and a set of abnormal fragments. The calculation module is used to perform temporal cumulative decoding on each valid data fragment in the valid fragment set along the spatial splicing trajectory of the valid data fragments converging towards the reconstruction center, so as to synthesize the recovered data at the reconstruction center; calculate the reconstruction content fingerprint of the recovered data, compare the reconstruction content fingerprint with the original content fingerprint, and output the evidence recovery verification result of the multimodal data.

[0065] The above embodiments use video data storage and recovery as an example for detailed explanation. The technical solution of the present invention is also applicable to other types of multimodal data storage and distributed recovery scenarios. For example: for image data, the object of original content fingerprint calculation is the complete byte stream of the image file (including file header and pixel data), and the fragmentation method also adopts erasure coding or direct file segmentation; for text or electronic contracts, the object of original content fingerprint calculation is the byte stream of text encoded in UTF-8; for mixed-modal data packets (such as packaged data of video, images and text associated with the same business event), they are regarded as a unified binary payload for unified content fingerprint calculation and fragmentation processing.

[0066] Regarding the implementation of trusted ledgers, in addition to the chained hash log structure used in this embodiment, consortium blockchain platforms (such as Hyperledger Fabric) or public blockchain evidence anchoring services can also be used (e.g., by writing the root hash value of the evidence chain into an Ethereum smart contract to achieve public trust anchoring). Regarding identity binding mechanisms, in addition to basic authentication based on account identifiers, identity binding based on PKI digital certificates can be introduced (when submitting data, the operating entity uses its private key to sign the content fingerprint, and the system uses the corresponding public key to verify the signature validity) or multi-factor authentication methods.

[0067] In terms of distributed storage and recovery, in addition to the erasure coding sharding method used in this embodiment, a multi-replica redundancy method can also be adopted (storing complete copies of the same data on storage nodes in three or more different geographical locations), or a combination of the two methods can be used (for example, using erasure coding sharding as the primary storage method, while additionally storing complete copies on critical nodes as a source for rapid recovery). Regarding recovery scenarios, in addition to platform-coordinated recovery automatically coordinated by the system, authorized collaborative recovery can also be supported: when the core scheduling capabilities of the central platform are limited, the system relies on the minimum recovery information pre-stored on each storage node (including shard metadata, inter-node communication credentials, and authorization tokens), and authorized administrators gradually reclaim shards and complete reconstruction verification through cross-node collaboration.

[0068] The above are preferred embodiments of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles described in the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A reliable multimodal data processing method, characterized in that, include: Obtain the raw content fingerprint of multimodal data; Following the temporal flow path of the operation in the trusted ledger, the original content fingerprint, the current operation subject identifier, the current operation timestamp, and the hash digest of the previous evidence node are chained hashed to generate the current operation evidence node, so as to form a continuous chain of operation evidence. When a data recovery command is triggered, the spatial distribution and flow trajectory of multiple candidate data shards in the distributed storage network are tracked to obtain multiple candidate data shards. Using the precise matching of fragment content fingerprints and the continuity of flow trajectory as classification boundaries, multiple candidate data fragments are verified and classified, and the multiple candidate data fragments are divided into a set of valid fragments and a set of abnormal fragments. For each valid data fragment in the valid fragment set, the spatial splicing trajectory of the valid data fragments converging towards the reconstruction center is used to perform temporal cumulative decoding on each fragment so as to synthesize the recovered data at the reconstruction center; Calculate the reconstructed content fingerprint of the recovered data, compare the reconstructed content fingerprint with the original content fingerprint, and output the evidence recovery verification result of the multimodal data, including: Based on the recovered data, it is divided into multiple data sub-blocks according to the preset dimension mapping rules, and the features of each data sub-block are mapped to a multi-dimensional feature space to obtain multiple data feature points; Based on multiple data feature points, calculate the spatial geometric distance between any two data feature points, and construct a spatial geometric topology graph based on the spatial geometric distance to obtain a geometric adjacency matrix; based on the geometric adjacency matrix, calculate the geometric invariants of the spatial geometric topology graph to obtain spatial geometric structure features, which include: Based on multiple data feature points, extract the multidimensional coordinate values ​​corresponding to any two data feature points, calculate the Euclidean distance between the multidimensional coordinate values, and obtain the set of spatial geometric distances between each pair of multiple data feature points. Based on the spatial geometric distance set, two data feature points that are less than a preset distance threshold are marked as adjacent nodes, node connection relationships are generated, and a spatial geometric topology map is obtained. Based on the node connection relationships in the spatial geometric topology graph, a square matrix is ​​constructed according to the node index to obtain the geometric adjacency matrix; based on the geometric adjacency matrix, the eigenvalues ​​of the geometric adjacency matrix are calculated to obtain the eigenvalue set; Based on the eigenvalue set, calculate the sum of squares of each eigenvalue in the eigenvalue set to obtain the first geometric invariant; Based on the eigenvalue set, the sum of the cubes of each eigenvalue in the eigenvalue set is calculated to obtain the second geometric invariant, where the eigenvalue set includes: Based on the node connection relationships in the spatial geometric topology graph, extract all node identifiers in the spatial geometric topology graph, and sort them in ascending order according to the preset identifier sorting rules to obtain an ordered node identifier sequence. Based on the ordered node identifier sequence, the total number of node identifiers is counted to obtain the matrix dimension parameters; Based on the matrix dimension parameter, generate an all-zero matrix whose dimension size matches the matrix dimension parameter to obtain the initial square matrix; Based on the node connection relationship, traverse each pair of adjacent nodes that have a connection relationship, determine the corresponding row index and column index in the initial square matrix, and update the element value at the corresponding position to the preset connection identifier value to obtain the geometric adjacency matrix. Based on the geometric adjacency matrix, and following the preset polynomial expansion rules, the determinant expansion of the difference between the main diagonal elements and the unknown variables is calculated to obtain the target characteristic polynomial. Based on the target feature polynomial, root search is performed according to the preset numerical iteration solution rules to obtain multiple candidate feature roots; Based on multiple candidate eigenvalues, each candidate eigenvalue is substituted into the target feature polynomial to calculate the residual, thus obtaining the residual value corresponding to each candidate eigenvalue. Based on the residual values ​​corresponding to each candidate eigenvalue, candidate eigenvalues ​​with residual values ​​less than a preset residual threshold are selected to obtain a set of eigenvalues; Based on the first geometric invariant and the second geometric invariant, the spatial geometric structure features are obtained by combining them according to the preset field concatenation rules. Based on the original byte stream and spatial geometric structure features of the recovered data, joint feature extraction is performed according to the preset cryptographic digest rules to obtain the reconstructed content fingerprint; Based on the reconstructed content fingerprint and the original content fingerprint, a character-level comparison is performed to obtain the fingerprint comparison difference value; Based on the fingerprint comparison difference value, determine whether the fingerprint comparison difference value is zero, and obtain the consistency judgment result; Based on the consistency judgment result, a corresponding data recovery trusted status identifier is generated, and the evidence recovery verification result is obtained.

2. The trusted multimodal data processing method as described in claim 1, characterized in that, To form a continuous chain of operational evidence, including: Based on the raw byte stream of multimodal data, feature extraction is performed according to preset cryptographic digest rules to obtain the raw data digest; Based on the original data digest, collision resistance verification and length standardization are performed to obtain the original content fingerprint; Based on the original content fingerprint, the current operation subject identifier, the current operation timestamp, and the hash digest of the previous evidence node, the fields are combined according to the preset time sequence splicing rules to obtain the current operation evidence element set. Based on the current set of operational evidence elements, secondary feature extraction is performed according to the preset cryptographic digest rules to obtain the current operational evidence node; Based on the current operational evidence node, it is appended to the end node of the trusted ledger to obtain a continuous chain of operational evidence.

3. The trusted multimodal data processing method as described in claim 2, characterized in that, Obtain multiple candidate data shards, including: Based on the data recovery instructions, the target data identifier is extracted; Based on the target data identifier, query the node routing table in the distributed storage network to obtain multiple candidate storage node identifiers corresponding to the target data; Based on multiple candidate storage node identifiers, a fragment acquisition request is sent to the corresponding candidate storage node, and multiple candidate data fragments are received and extracted. Based on multiple candidate data fragments, the storage node location information and transmission path node information corresponding to each candidate data fragment are extracted to obtain the spatial distribution status and flow trajectory of multiple candidate data fragments.

4. The trusted multimodal data processing method as described in claim 3, characterized in that, Using precise matching of fragment content fingerprints and the continuity of flow trajectories as classification boundaries, multiple candidate data fragments are validated and classified, dividing them into a valid fragment set and an abnormal fragment set, including: Based on multiple candidate data slices, feature extraction is performed according to preset cryptographic digest rules to obtain the slice content fingerprints corresponding to the multiple candidate data slices; Based on the fragment content fingerprint, a precise matching and comparison is performed with the original fragment fingerprint database corresponding to the target data to obtain the fragment content fingerprint matching result. Based on the flow trajectory, extract the timestamps of adjacent path nodes during the transmission process of each candidate data fragment, calculate the time interval between adjacent path nodes, and obtain the continuity verification result of the flow trajectory. Based on the fingerprint matching results of the fragment content and the continuity verification results of the flow trajectory, the classification is verified using the accurate matching of the fragment content fingerprint and the continuity of the flow trajectory as the classification boundary, and the classification labels corresponding to multiple candidate data fragments are obtained. Based on the classification labels, multiple candidate data shards are divided into a set of valid shards and a set of abnormal shards.

5. The trusted multimodal data processing method as described in claim 4, characterized in that, Based on the transmission trajectory, the timestamps of adjacent path nodes during the transmission process of each candidate data fragment are extracted, the time interval between adjacent path nodes is calculated, and the continuity verification result of the transmission trajectory is obtained, including: Based on the flow trajectory, the identifiers of each path node that each candidate data fragment passes through during the transmission process are extracted to obtain the path node identifier sequence; Based on the path node identifier sequence, query the transmission log of the corresponding node, extract the data reception timestamp and data transmission timestamp corresponding to each path node, and obtain the set of timestamps of adjacent path nodes; Based on the set of timestamps of adjacent path nodes, calculate the time difference between the data transmission timestamp of the previous node and the data reception timestamp of the next node in the adjacent path nodes to obtain the set of time intervals between adjacent path nodes. Based on the set of time intervals, each time interval is compared with a preset normal transmission delay threshold to obtain the delay determination result corresponding to each time interval. Based on the delay determination results, the proportion of time intervals with normal delay determination results to the total number of time intervals is calculated to obtain the continuity verification results of the flow trajectory.

6. The trusted multimodal data processing method as described in claim 5, characterized in that, For each valid data fragment in the valid fragment set, the spatial stitching trajectory converges from the valid data fragment to the reconstruction center, and temporal cumulative decoding is performed fragment by fragment to synthesize the recovered data at the reconstruction center, including: Based on the set of valid data shards, extract the shard number and shard payload data corresponding to each valid data shard to obtain the shard metadata set. Based on the fragment sequence number in the fragment metadata set, query the preset fragment location mapping table to obtain the target offset position corresponding to each valid data fragment; Based on the target offset position, sort each valid data fragment in ascending order of target offset position to obtain an ordered fragment queue; Based on the ordered fragment queue, the fragment payload data of each valid data fragment is extracted sequentially and spliced ​​and accumulated according to the corresponding target offset position to obtain the initial reconstructed data stream; Based on the initial reconstructed data stream, data error correction and integrity repair are performed according to the preset erasure coding decoding rules to obtain the recovered data.

7. A trusted multimodal data processing system, characterized in that, The system is used to perform the method as described in any one of claims 1 to 6, comprising: The acquisition module is used to acquire the original content fingerprint of multimodal data; along the temporal flow path of the operation behavior in the trusted ledger, the original content fingerprint, the current operation subject identifier, the current operation timestamp, and the hash digest of the previous evidence node are chained hashed to generate the current operation evidence node, so as to form a continuous operation evidence chain. The judgment module is used to track the spatial distribution and flow trajectory of multiple candidate data fragments in the distributed storage network when a data recovery command is triggered, and obtain multiple candidate data fragments; using the accurate matching of fragment content fingerprints and the continuity of flow trajectory as classification boundaries, the multiple candidate data fragments are verified and classified into a set of valid fragments and a set of abnormal fragments. The calculation module is used to perform temporal cumulative decoding on each valid data fragment in the valid fragment set along the spatial splicing trajectory of the valid data fragments converging towards the reconstruction center, so as to synthesize the recovered data at the reconstruction center; calculate the reconstruction content fingerprint of the recovered data, compare the reconstruction content fingerprint with the original content fingerprint, and output the evidence recovery verification result of the multimodal data.

Images