Multi-modal data cross-domain secure evidence chain construction system based on federated learning

Abstract

The application belongs to the technical field of data security, and relates to a multi-modal data cross-domain security evidence chain construction system based on federated learning, which comprises the following modules: a multi-modal data acquisition and feature mapping module, which generates cross-modal shared hidden space feature vectors and cross-modal mutual information entropy characteristic values; a gradient calculation and adaptive encryption module, which aggregates to generate encrypted gradient ciphertext blocks; a homomorphic aggregation and blockchain beacon generation module, which generates a global smart contract beacon; a dynamic token generation and analysis module, which aggregates and encapsulates to generate a dynamic retrieval token; a knowledge graph connection decision and construction module, which generates a multi-modal evidence chain knowledge graph network; and an evidence chain compensation and traceability output module, which aggregates to generate a cross-domain security evidence chain. The application solves the problem of high computational delay defects caused by static high-intensity encryption and complex consensus verification in the existing cross-platform privacy collaboration mechanism, which leads to the problem of being unable to balance the dynamic balance between privacy protection strength and system real-time processing efficiency.

Description

Technical Field

[0001] This invention belongs to the technical field of data security and relates to a multimodal data cross-domain secure evidence chain construction system based on federated learning. Background Technology

[0002] In today's booming digital social landscape, cross-platform social interaction has become a crucial way for people to exchange information. The massive amounts of user data typically utilize centralized storage architectures, leading to high data concentration and weakening users' control over their own data, which can easily trigger data breaches and security risks. Federated learning, as an emerging technology, allows participating parties to jointly model data without sharing the original data, providing a technical path for cross-platform data collaborative utilization. Meanwhile, blockchain technology, with its decentralized and traceable characteristics, offers solutions to some extent for secure data storage and trusted sharing.

[0003] Regarding the issue of collaborative data privacy protection in cross-platform social networking, existing technical solutions, such as patent application number 202510713102.6, disclose a cross-platform social privacy collaborative protection system based on federated learning and blockchain. This scheme generates global model initialization parameters through a multi-party secure aggregation algorithm, performs layered encryption processing on the original data using homomorphic encryption and zero-knowledge proofs, and utilizes a hash time-lock protocol to achieve cross-platform data synchronization. Furthermore, the system verifies model updates using an improved Byzantine fault-tolerant algorithm that dynamically adjusts voting weights based on node reputation values, and triggers privacy protection rules based on a differential privacy noise injection algorithm.

[0004] While the aforementioned existing technologies construct a multi-layered protection system for cross-platform privacy, they have certain limitations in complex cross-platform interaction scenarios. Homomorphic encryption combined with a layered encryption mechanism based on zero-knowledge proofs and a complex Byzantine fault-tolerant consensus algorithm with node reputation evaluation enhances data security but introduces significant computational overhead and communication latency. This fixed multi-layered encryption and complex consensus process struggles to adapt to real-time network conditions and node computing capabilities when dealing with massive amounts of high-concurrency social data, easily leading to a decline in overall system processing efficiency and making it difficult to balance strong privacy protection with real-time system response performance.

[0005] Therefore, how to overcome the high computational latency caused by static high-strength encryption and complex consensus verification in existing cross-platform privacy collaboration mechanisms, which makes it impossible to achieve a dynamic balance between the strength of privacy protection and the real-time processing efficiency of the system. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a multimodal data cross-domain secure evidence chain construction system based on federated learning.

[0007] A multimodal data cross-domain secure evidence chain construction system based on federated learning includes: The multimodal data acquisition and feature mapping module acquires multimodal physical sensing flow, extracts high-dimensional temporal feature matrix from preset feature encoding layer, performs cross-modal feature distance compression by calculating cross-head alignment correlation, and generates cross-modal shared latent space feature vector and cross-modal mutual information entropy feature value. The gradient calculation and adaptive encryption module performs backpropagation operations based on local business model parameters to extract basic feature gradient streams and cross-modal decision gradient streams. It imports cross-modal mutual information entropy feature values ​​to adjust the noise variance of the basic feature gradient streams and improves the encryption bit length barrier level of the cross-modal decision gradient streams, and aggregates them to generate encrypted gradient ciphertext blocks. The homomorphic aggregation and blockchain beacon generation module aggregates encrypted gradient ciphertext blocks to a preset trusted central node to perform homomorphic ciphertext feature aggregation, strips and decrypts to generate covering parameters, extracts feature hash digest values ​​and trusted central physical timestamps and inputs them into the cross-domain consortium blockchain network to generate a global smart contract beacon. The dynamic token generation and parsing module intercepts the global smart contract beacon and extracts the block generation time identifier. It performs serialization and concatenation calculations on the cross-modal shared latent space feature vector and the block generation time identifier to generate a fusion operation base string. Then, it generates a digest feature through a one-way hashing and obfuscation algorithm and independently encapsulates it to generate a dynamic retrieval token. The knowledge graph connection adjudication and construction module receives graph connection requests carrying dynamic retrieval tokens, extracts summary features from the dynamic retrieval tokens, performs hash collision verification based on pre-stored cross-modal shared latent space feature vectors and candidate block generation time identifiers traversed within a preset time window, matches and restores the block generation time identifiers, cross-verifies the preset hardware acquisition physical time sequence and block generation time identifiers to calculate the spatiotemporal decay penalty weight, cuts connection operations with weights lower than the preset security limit, and generates a multimodal evidence chain knowledge graph network. The evidence chain compensation and traceability output module uses a preset cross-modal high-quality anchor point feature set to guide the incomplete retrieval features to be reshaped into edge high-dimensional representation vectors. Using the edge high-dimensional representation vectors as core pointers, cross-comparison is performed along the multimodal evidence chain knowledge graph network to generate a cross-domain secure evidence chain.

[0008] A further aspect of the present invention involves taking multimodal physical sensing flow, including the following steps: Activate distributed IoT edge computing nodes; Simultaneously acquire continuous time-series video images and structured logs of the communication network at the edge coverage site; Continuous video time-series images are merged and integrated with structured logs from communication networks to construct a multimodal physical sensing stream.

[0009] A further aspect of this invention involves generating cross-modal shared latent space feature vectors and cross-modal mutual information entropy feature values, comprising the following steps: Constructing multi-head alignment correlations for high-dimensional temporal feature matrices; Calculate the local information overlap and crossover degree between multiple feature distribution databases; The heterogeneous signal stream is subjected to cross-modal feature distance compression, and the output is a cross-modal shared latent space feature vector with a unified feature dimension and a cross-modal mutual information entropy feature value.

[0010] A further aspect of this invention involves importing cross-modal mutual information entropy feature values ​​to adjust the noise variance of the basic feature gradient flow and improving the encryption bit length barrier level of the cross-modal decision gradient flow, including the following steps: Import cross-modal mutual information entropy feature values ​​as reverse compensation control factors and inject them into the underlying differential privacy generation module; Based on the changing trend of multi-physical flow coupling, noise variance is applied to the Laplace distribution of the basic feature gradient flow in real time. The redundant computing power released by reducing noise addition operations is extracted and sent to the improved homomorphic cryptographic processor, which forcibly increases the encryption bit length barrier level when processing cross-modal decision gradient flow.

[0011] A further aspect of the present invention involves aggregating and generating encrypted gradient ciphertext blocks, comprising the following steps: The cross-modal mutual information entropy eigenvalues ​​are converted into ciphertext verification preimages through irreversible mathematical folding. The basic feature gradient flow after adjusting the noise variance, the cross-modal decision gradient flow after increasing the encryption bit length barrier level, and the ciphertext verification preimage are encapsulated. Aggregate and encapsulate the output encrypted gradient ciphertext block.

[0012] A further aspect of this invention involves extracting the feature hash digest value and the trusted center's physical time stamp, inputting them into a cross-domain consortium blockchain network, and generating a global smart contract beacon, including the following steps: Extract the feature hash digest value and the trusted center physical time tag generated in this update cycle; The concatenated feature hash digest value and the trusted center physical time tag are entered into the secure data ledger of the cross-domain consortium blockchain network to complete the verification consensus packaging; A global smart contract beacon containing a verifiable block generation timestamp is transmitted back via a tamper-proof mainnet link.

[0013] A further aspect of this invention involves intercepting the global smart contract beacon and extracting the block generation time identifier, comprising the following steps: The coverage of the parameter matrix iteration of each physical domain edge workstation in the access system in a new cycle is detected; Intercepting and dismantling global smart contract beacons actively broadcast by cross-domain consortium blockchain networks; The application system-level parser filters out irrelevant verification words and forcibly removes the block generation time stamp and global Merkle tree root node data from the reconstructed data stream.

[0014] A further aspect of this invention involves performing serialization and concatenation calculations on the cross-modal shared latent space feature vector and the block generation time identifier to generate a fusion operation base string, and then generating a digest feature through a one-way hashing and obfuscation algorithm. This is then independently encapsulated to generate a dynamic retrieval token, comprising the following steps: Perform serialization concatenation calculation on the cross-modal shared latent space feature vectors that reside in the local cache pool for a long time and the block generation time identifier; Complete the direct concatenation to generate the basic string for fusion operation; The fusion operation base string is fed into the one-way hashing and obfuscation algorithm conversion module to output a digest value with highly variable characteristics; The digest value is encapsulated independently to generate a dynamic retrieval token that serves as an anti-tampering verification identifier.

[0015] A further aspect of this invention involves cross-verifying the preset hardware acquisition physical timing and block generation time identifier to calculate the spatiotemporal attenuation penalty weight, including the following steps: Obtain the hardware acquisition physical timing sequence reported by the edge layer nodes of the graph; Extract the digest value from the received dynamic retrieval token; Based on the pre-stored cross-modal shared latent space feature vector and the candidate block generation time identifier traversed within a reasonable time window, serialization and splicing are performed, and then fed into the one-way hash obfuscation algorithm conversion module to perform hash calculation; When the hash value of the calculated output collides with the digest value, the successfully matched candidate block generation time identifier is confirmed as the block generation time identifier that has been solved and restored. Cross-verify the deviation scale range between the physical timing of hardware acquisition and the block generation time identifier reconstructed by calculation; The physical time lag error reflected by the above time stamp decoding comparison is calculated and output as an exponentially decreasing spatiotemporal decay penalty weight.

[0016] A further aspect of this invention utilizes a pre-defined cross-modal high-quality anchor point feature set to guide the reshaping of incomplete retrieval features into a high-dimensional edge representation vector, comprising the following steps: Stimulate the edge-cloud collaborative teacher-student network compensation module guided and driven by a preset cross-modal high-quality anchor point feature set; Incomplete retrieval features that have suffered from confidence degradation due to differential noise interference in edge device execution layer localization; The incomplete retrieval features are guided to overcome the signal-to-noise ratio abyss and be re-projected onto the high-dimensional net feature plane to generate edge high-dimensional representation vectors.

[0017] In summary, the present invention has the following beneficial technical effects: 1. A cross-modal mutual information entropy feature is introduced to dynamically adjust the noise injection intensity of differential privacy and the bit length barrier level of homomorphic encryption. The system transforms the statistical coupling degree between multimodal signals into an adaptive protection strategy: when the correlation between multimodal data is high, the system automatically reduces the noise intensity to retain more shared features and simultaneously improves the encryption security level of the gradient flow for cross-modal fusion judgment. This mechanism effectively reduces the unnecessary loss of high-value evidence features while ensuring the local privacy of the original sensor data, achieving a dynamic balance between privacy protection strength and feature fidelity.

[0018] 2. The encrypted gradient ciphertext blocks generated by each edge node are aggregated into homomorphic ciphertexts at the trusted central node after anonymization, and a near-end deviation penalty mechanism is introduced. Simultaneously, combined with the global smart contract beacon of the cross-domain consortium blockchain, the block generation time is used as the consensus time anchor. This mechanism binds the model update process to physical time, enabling the reconstruction of global parameters without exposing the original update content of the nodes. This not only achieves anti-counterfeiting and traceability of the federated learning iteration cycle but also provides a tamper-proof temporal basis for subsequent cross-domain mutual verification of evidence.

[0019] 3. By obfuscating and hashing the block generation time stamp with the local multimodal shared latent space feature vector, a dynamic retrieval token is generated. The time synchronization deviation implicit in the token is used to implement spatiotemporal decay penalties. This mechanism can effectively verify the temporal authenticity of cross-node connection requests in the knowledge graph, automatically cutting off requests with time differences exceeding a safety threshold, and granting write permissions only to nodes with high spatiotemporal consistency. This helps prevent the risk of establishing false evidence associations through replaying expired tokens or forging tokens, thereby constructing a reliable multimodal evidence network with physical temporal constraints.

[0020] 4. In the post-confirmation stage of the evidence chain, to address the issue of signal-to-noise ratio attenuation and confidence decline in some features due to differential privacy noise, this technical solution introduces an edge-cloud collaborative teacher-student network compensation module guided by a global high-quality anchor feature set. This module reshapes and reprojects degraded features onto a high-dimensional net feature plane and performs multi-level leap-style cross-comparison along the approval boundaries with high weights in the knowledge graph network. Through this feature compensation and cross-comparison mechanism, originally isolated multimodal clues that cross regions and systems can be transformed into a cross-domain evidence chain with logical closure, effectively reducing evidence chain breaks caused by feature degradation and improving the overall integrity of the evidence chain. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings are used to provide a further understanding of the present invention.

[0022] Figure 1 This discloses a schematic diagram of the framework in the embodiments of this application.

[0023] Figure 2 This discloses a flowchart of an embodiment of this application. Detailed Implementation

[0024] The following is in conjunction with the appendix Figure 1 - Figure 2 A preferred description of the present invention is provided below.

[0025] See attached document Figure 1 - Figure 2 This invention proposes a multimodal data cross-domain secure evidence chain construction system based on federated learning, comprising the following modules: The multimodal data acquisition and feature mapping module acquires multimodal physical sensing flow, extracts high-dimensional temporal feature matrix from preset feature encoding layer, performs cross-modal feature distance compression by calculating cross-head alignment correlation, and generates cross-modal shared latent space feature vector and cross-modal mutual information entropy feature value. The gradient calculation and adaptive encryption module performs backpropagation operations based on local business model parameters to extract basic feature gradient streams and cross-modal decision gradient streams. It imports cross-modal mutual information entropy feature values ​​to adjust the noise variance of the basic feature gradient streams and improves the encryption bit length barrier level of the cross-modal decision gradient streams, and aggregates them to generate encrypted gradient ciphertext blocks. The homomorphic aggregation and blockchain beacon generation module aggregates encrypted gradient ciphertext blocks to a preset trusted central node to perform homomorphic ciphertext feature aggregation, strips and decrypts to generate covering parameters, extracts feature hash digest values ​​and trusted central physical timestamps and inputs them into the cross-domain consortium blockchain network to generate a global smart contract beacon. The dynamic token generation and parsing module intercepts the global smart contract beacon and extracts the block generation time identifier. It performs serialization and concatenation calculations on the cross-modal shared latent space feature vector and the block generation time identifier to generate a fusion operation base string. Then, it generates a digest feature through a one-way hashing and obfuscation algorithm and independently encapsulates it to generate a dynamic retrieval token. The knowledge graph connection adjudication and construction module receives graph connection requests carrying dynamic retrieval tokens, extracts summary features from the dynamic retrieval tokens, performs hash collision verification based on pre-stored cross-modal shared latent space feature vectors and candidate block generation time identifiers traversed within a preset time window, matches and restores the block generation time identifiers, cross-verifies the preset hardware acquisition physical time sequence and block generation time identifiers to calculate the spatiotemporal decay penalty weight, cuts connection operations with weights lower than the preset security limit, and generates a multimodal evidence chain knowledge graph network. The evidence chain compensation and traceability output module uses a preset cross-modal high-quality anchor point feature set to guide the incomplete retrieval features to be reshaped into edge high-dimensional representation vectors. Using the edge high-dimensional representation vectors as core pointers, cross-comparison is performed along the multimodal evidence chain knowledge graph network to generate a cross-domain secure evidence chain.

[0026] In one embodiment of the present invention, the multimodal data acquisition and feature mapping module is used to perform the following steps: Construct multi-head alignment correlation of high-dimensional temporal feature matrix; calculate local information overlap and cross-degree between multiple feature distribution libraries; perform cross-modal feature distance compression on heterogeneous signal streams, and map out cross-modal shared latent space feature vectors and cross-modal mutual information entropy feature values ​​with unified feature dimensions.

[0027] Specifically, in one embodiment of the present invention, the method for constructing a cross-domain secure evidence chain for multimodal data based on federated learning is first executed by a local processing system deployed on a distributed edge computing node, which performs the multimodal data acquisition and feature mapping module. The local processing system, through its built-in resource scheduling and task execution module, first activates multiple sensors physically or wirelessly connected to the edge computing node, including but not limited to high-definition network cameras and communication network routers or switches.

[0028] The system triggers data acquisition with a preset synchronization clock signal, acquiring a continuous video time-series image stream conforming to the Real-Time Streaming Protocol (RTSP) from a high-definition network camera, and simultaneously acquiring a structured log information stream conforming to the Syslog protocol or outputting in the lightweight data exchange format JSON from the management interface of the communication network router or switch. The data integration unit of the local processing system parses the timestamp information in the headers of the two data streams, packages and encapsulates the video frame sequences and log entries with timestamp alignment and errors within a predetermined acquisition period, constructing a logically unified multimodal physical sensor stream. The sensor stream is essentially a composite data structure marked with timestamp synchronization, such as a combination of video frame sequences and network log records.

[0029] The multimodal physical sensing flow is imported into the heterogeneous feature representation layer built into the local computation model for decoupling and encoding. For video temporal image modalities, the system segments the data into segments with a fixed time window length T1, such as 1 s or 2 s. The consecutive image frames within each segment are stacked into a video data block, which is then input into a pre-trained spatiotemporal convolutional neural network, such as a 3D convolutional C3D or dilated 3D convolutional I3D type. The feature maps that can represent dynamic spatiotemporal events output by the intermediate layers of the network are extracted as its high-dimensional temporal feature matrix, such as a 512-dimensional feature vector sequence.

[0030] For the structured log modality of communication networks, the system performs preprocessing on the log text, such as word segmentation and stop word removal, and then inputs it into a pre-trained natural language processing model. The natural language processing model is of the type of bidirectional encoder representation based on transformer (BERT) or robust optimization BERT pre-training method (RoBERTa). The model outputs context embedding vectors that can represent the semantics of the text as its high-dimensional temporal feature matrix.

[0031] Subsequently, the local processing system initiates a multi-head cross-modal alignment module. This module receives high-dimensional temporal feature matrices from different modalities and maps them to a cross-modal shared latent space of the same dimension through trainable linear projection layers. Within this shared latent space, a multi-head attention mechanism calculates the correlation weights between the features of the two modalities. After weighted aggregation, the results are output through a fully connected network layer, generating a cross-modal shared latent space feature vector representing the core semantics after multimodal signal fusion. This vector is a feature vector representing the aligned representation of multimodal information in the feature space. Its dimension D is a preset hyperparameter, typically set to 256 or 512. The setting is based on engineering practice to balance representational power and computational cost. If the dimension is set below 256, it can easily lead to severe information loss and feature aliasing of multimodal high-dimensional features, such as small movements and jumps in video sequences, during dimensionality reduction projection. If the dimension is higher than 512, the resource overhead of subsequent homomorphic encryption and XOR obfuscation calculations will increase exponentially, thereby paralyzing the millisecond-level real-time response capability of edge computing nodes. Setting it to 256 or 512 is a balance achieved between feature fidelity and edge computing power.

[0032] Meanwhile, to quantify the statistical dependence between the two modes, the system estimates their joint probability distribution and marginal probability distribution based on the quantized feature distributions of the high-dimensional time-series feature matrices of the two modes. Based on these probability distributions, the system calculates the discrete mutual information between the two modes, ultimately generating a single numerical value for the cross-modal mutual information entropy feature. This cross-modal mutual information entropy feature is a scalar value used to measure the statistical correlation between signals from different modes. The specific calculation of this cross-modal mutual information entropy feature is shown in the following formula:

[0033] In the formula, This represents the mutual information between the video modality and the log modality; and These represent the sets of states that represent all possible values ​​of video features and log features within the same time window, respectively. and They represent sets respectively and The value state of a specific feature in; Within the same time window, the video feature values ​​are... And the log feature value is The joint probability is obtained through statistical analysis of a large amount of historical sample data or through methods such as kernel density estimation; and The video feature values ​​are respectively And log feature values The marginal probability is obtained in a similar way to the joint probability, both based on prior statistics of large-scale datasets. To ensure mathematical completeness, based on the general principles of information theory, if the feature state does not appear, causing the joint probability or marginal probability to be 0, the accumulation of that term is ignored, i.e., it is agreed that... And avoids the mathematical anomaly of a denominator of zero; The base can be 2, e, or 10, corresponding to the units of mutual information: bit, nat, or hartley, respectively. In this scheme, it is set to 2, with bits as the unit, which is a common practice in the field of information theory.

[0034] For example, suppose an edge computing node acquires a 1-second video clip with a frame rate of 25fps and two related network logs at time T. The system inputs the tensor formed by these 25 frames into a pre-trained C3D network to obtain a 512-dimensional video feature matrix. V’ . Translate two log texts, for example, " login failed" and "Port scandetected from "Inputting the BERT model yields a 768-dimensional log feature matrix L'."

[0035] Local processing system will V’ Through a 512x256 dimensional projection matrix Projection yields a 256-dimensional vector. ,Will L’ Through a 768x256 dimensional projection matrix Projection yields a 256-dimensional vector. Calculated through attention mechanisms and The weighted average is then passed through a 256x256 fully connected layer to finally generate a 256-dimensional cross-modal shared latent space feature vector, for example, with values ​​of [0.85, -0.21, ..., 0.55].

[0036] To calculate the cross-modal mutual information entropy feature value, assuming that the video features have been quantized... V and log features L Each has two possible states {high risk, low risk}. By analyzing a historical database containing 1000 samples, the joint probability distribution of the two is statistically obtained, for example, p( V =High, L =height)=0.2, p( V =High, L =low)=0.1, p( V =low, L =high)=0.05,p( V =low, L =low)=0.65. Therefore, the marginal probability p( V =height)=0.3, p( L =High)=0.25. Substitute these probability values ​​into the mutual information formula above for calculation. For example, calculate one term: p(high,high)log2(p(high,high) / p(high)p(high))=0.2log2(0.2 / (0.3×0.25))≈0.283bit. Summate the calculation results of all four combinations to obtain a final value, for example... M ( V ; L =0.45bit. This value constitutes the cross-modal mutual information entropy feature value output in this step.

[0037] In one embodiment of the present invention, the gradient calculation and adaptive encryption module is used to perform the following steps: The cross-modal mutual information entropy feature values ​​are converted into ciphertext verification preimages through irreversible mathematical folding; the basic feature gradient flow after adjusting the noise variance, the cross-modal decision gradient flow after increasing the encryption bit length barrier level, and the ciphertext verification preimages are encapsulated; the encapsulated and aggregated output encryption gradient ciphertext blocks are then generated.

[0038] Specifically, after completing the multimodal data acquisition and feature mapping modules, the local processing system connects to the gradient calculation and adaptive encryption modules to respond to the global training scheduling of the federated learning center. The local processing system first initiates the backpropagation algorithm based on the loss function value calculated by the local model on the current mini-batch of samples. During this process, the system performs layered interception and extraction of the gradient flow in the local computation graph.

[0039] On the one hand, gradient hooks are set at the end of the convolutional or embedding layers of basic feature extraction network models such as C3D or BERT to capture and aggregate the basic feature gradient flow. The basic feature gradient flow is the partial derivative of the loss function with respect to the underlying parameters, carrying the pattern update information of the original input data.

[0040] On the other hand, gradient hooks are also set at the trainable projection layer and attention weight matrix in the multi-head cross-modal alignment module to capture and aggregate the cross-modal decision gradient flow that reflects the cross-modal fusion logic decision bias. This cross-modal decision gradient flow is the partial derivative of the loss function with respect to the parameters of the high-level modal fusion module, reflecting the error of the model in learning how to associate different modal information.

[0041] After acquiring the two gradient streams, the system initiates an asymmetric privacy-preserving encapsulation process, using the cross-modal mutual information entropy feature value obtained from the multimodal data acquisition and feature mapping module as the core adaptive modulation input. The underlying differential privacy generation module is activated, importing the cross-modal mutual information entropy feature value and dynamically adjusting the intensity of the Laplace noise added to the basic feature gradient stream based on the magnitude of this feature value. Adding noise to the Laplace distribution here is a common mechanism that satisfies the definition of differential privacy, providing mathematically provable privacy protection by adding random noise extracted from the Laplace distribution to the real data.

[0042] A higher cross-modal mutual information entropy value indicates a stronger correlation between the video and log modalities, representing high-quality evidence data. Therefore, the system continuously attenuates the applied Laplace distribution noise variance to preserve the gradient information of key features as much as possible. Conversely, a lower mutual information entropy value indicates a less clear correlation between the currently collected data from the two modalities. To protect potentially accidental coupled data, the system increases the noise variance to preserve the gradient information of key features as much as possible. The specific scale parameter for the adaptive noise variance attenuation of the Laplace noise applied to the gradient flow of basic features is defined. Determined by the following formula:

[0043] In the formula, The Laplace noise scale parameter to be applied at present; The maximum noise level is preset, based on the overall privacy budget that the system needs to meet. The setting, for example, is set to 1.0 to ensure privacy in the worst-case scenario, based on the fallback protection mechanism of the global privacy budget of the federated learning system. This ensures that in the worst-case scenario where mutual information entropy approaches zero (i.e., extreme uncorrelation between modes), the added large noise can disrupt and mask the true gradient distribution of the original data of a single edge node, effectively resisting reverse inference attacks targeting local physical logs. This represents the current mutual information entropy value obtained from the cross-modal mutual information entropy feature value.

[0044] The preset attenuation coefficient is determined based on the relationship between model performance and privacy protection obtained from experimental statistics, and typically ranges from 0.5 to 1.5. If the value is less than 0.5, the rate at which noise decays with increasing mutual information is too slow, causing redundant noise to be added to the features of highly correlated evidence data, making it difficult for the global graph model to converge; if If the value is greater than 1.5, the noise attenuation will drop sharply, posing a risk of privacy exposure of local high-frequency signals. Setting it to 1.0 ensures high-confidence connections at nodes in the evidence chain graph while smoothing the transition range of the privacy protection barrier. This embodiment assumes a scenario that prioritizes preserving highly relevant evidence, and its physical unit is set to... This is to offset the difference in mutual information by ensuring that the exponent is a dimensionless pure number.

[0045] The system implements a policy-based computing power reallocation mechanism. This mechanism quantifies the equivalent index of redundant computing power extracted by reducing the intensity of noisy computations and maps this index to an encryption level upgrade instruction, which is then sent to the improved homomorphic cryptographic processor. Upon receiving the cross-modal decision gradient stream, the processor selects a higher-order encryption bit-length barrier level based on the incoming encryption level upgrade instruction, for example, increasing the default 512-bit key length to 1024 bits or 2048 bits, and performs homomorphic encryption on the cross-modal decision gradient stream. The encryption bit-length barrier level refers to the bit length of the public key in the homomorphic encryption scheme and is a key indicator of its resistance to attacks. For homomorphic encryption of the cross-modal decision gradient stream, the encryption bit-length barrier level... Determined by the following formula:

[0046] In the formula, This should be the encryption bit length used currently; It is the basic minimum encryption bit length, usually 512 bits, which can provide basic security. It is the bit length increment for each encryption level, and is usually set to 512 bits; This is the step size of the mutual information entropy decrease required to trigger an upgrade in encryption level. As a hyperparameter for adjusting the sensitivity of encryption level upgrades, it is set to 0.2 bits in this scheme. The choice of 0.2 bits as the step size is because this threshold matches the statistical sensitivity range of mutual information entropy state transitions in actual quantization calculations. If the step size is too small, such as less than 0.1, it will cause the edge processor to frequently switch the encryption key bit length from 512 to 4096, resulting in severe computing power jitter and communication latency. If the step size is too large, it will lead to failure to upgrade the encryption barrier level in time when encountering strongly correlated core evidence features, posing a risk of data interception and cracking. This refers to the calculated mutual information entropy value. This is the floor function.

[0047] Next, the cross-modal mutual information entropy feature value, the edge node's unique device identifier, and the local timestamp of the current round are concatenated into a string. After salting to prevent hash collisions, the string is fed into the SHA-256 secure hash algorithm module. Through an irreversible mathematical folding process, it is converted into a fixed-length hash value as the ciphertext verification preimage. Irreversible mathematical folding usually refers to the process of using cryptographic hash functions such as SHA-256. The ciphertext verification preimage is the hash digest generated from this process, used for subsequent integrity and provenance verification. The local processing system aggregates and encapsulates the basic feature gradient stream perturbed by Laplace noise, the homomorphically encrypted cross-modal decision gradient stream, and the ciphertext verification preimage in a standardized data structure, and outputs the encrypted gradient ciphertext block.

[0048] For example, taking the output of the preceding multimodal data acquisition and feature mapping module as an example, after receiving the instruction from the federated server, the local processing system begins to execute backpropagation. Assume the currently calculated basic feature gradient flow is a 2x2 matrix. The cross-modal gradient flow is determined as a 3-dimensional vector. .

[0049] The obtained cross-modal mutual information entropy eigenvalue is M = 0.45 bits. The value applied to... The Laplace noise scale. Assume... , Then the noise scale 638. The differential privacy generation module generates a Laplace noise matrix with a mean of 0 and a scale of 0.638, for example... Add it to get .

[0050] Sure The encryption bit length, assuming , , Then the length The improved homomorphic cryptography processor then employs a... Paillier public key pair Encryption is performed to obtain ciphertext. Then, the system... M The 0.45-bit value is used to perform a SHA-256 operation, generating a ciphertext verification preimage, such as "8a1f…c3d9". The system will then process the preimage. Encrypted The encrypted verification preimage is integrated and packaged to generate an encrypted gradient ciphertext block, which is then prepared to be sent to the central node.

[0051] In one embodiment of the present invention, the homomorphic aggregation and blockchain beacon generation module is used to perform the following steps: Extract the feature hash digest value and trusted center physical time tag generated in this update cycle; concatenate the feature hash digest value and trusted center physical time tag and enter them into the secure data ledger of the cross-domain consortium blockchain network to complete the verification consensus packaging; transmit back the global smart contract beacon containing the verifiable block generation timestamp through the tamper-proof mainnet link.

[0052] Specifically, gradient calculation and adaptive encryption modules are completed locally on each edge computing node. After generating the corresponding encrypted gradient ciphertext blocks, they enter the homomorphic aggregation and blockchain beacon generation modules. The preset trusted central node is the central server that undertakes the tasks of federated learning model aggregation calculation and blockchain interaction. Its trustworthiness is guaranteed by the deployment environment and management strategies. The trusted central node is usually deployed in a physically isolated core data center with a high level of security, and it initiates the data aggregation service.

[0053] The data aggregation service receives data by allocating a private, isolated communication link using TLS 1.3 encryption to each legitimate edge node. This means it's a logically or physically independent data transmission channel built outside the public network. Each independent physical domain then unidirectionally uploads its generated encrypted gradient ciphertext blocks to the trusted central node via this link.

[0054] After receiving ciphertext blocks from a large number of nodes, the processor of the trusted central node verifies the ciphertext verification preimage in each ciphertext block to confirm its source legitimacy and data integrity. Once the verification passes, the processor directly performs ciphertext state aggregation operations on the homomorphic encrypted portions contained in all ciphertext blocks.

[0055] Specifically, additive homomorphic aggregation is performed on the cross-modal decision gradient stream ciphertexts received from different nodes and encrypted using the same high-order homomorphic public key. Homomorphic ciphertext feature aggregation refers to performing operations on the encrypted data without decryption, and ensuring that the aggregation result after decryption is consistent with the result of first aggregating and then encrypting the plaintext data. Element-wise accumulation and averaging are also performed on the basic feature gradient streams of all ciphertext blocks that have been perturbed by differential privacy noise. Since they are not homomorphically encrypted, but for the sake of uniform processing, this is represented by averaging the corresponding elements of each gradient matrix.

[0056] This aggregation process introduces a proximal term deviation penalty mechanism, a technique used in distributed optimization to smooth model updates and enhance learning convergence. This mechanism uses a regularization term to penalize local models for deviating too far from the global model. Here, it is extended to monitor gradient deviation; that is, while performing aggregation computation, the norm distance of the gradient contribution of an individual node relative to the current global aggregation result is monitored. If the gradient of any node deviates from the average trend of the group by more than a preset trend threshold, the penalty is applied. This trend threshold By statistically analyzing historical model update data and setting the 99th percentile of its gradient norm distribution, the aggregator dynamically reduces the aggregation weight of that node, thereby suppressing the excessive influence of malicious or abnormal gradients on the global model.

[0057] After the encrypted aggregation is completed, the Trust Center uses its homomorphic private key to decrypt the aggregation result in one go to obtain the covering parameters. These covering parameters are the full model update gradients reconstructed from the gradients of all or most participating nodes globally. This gradient is not applied directly; instead, a portion of it is extracted to construct the update instructions for the model matrices issued to each subordinate local device.

[0058] Simultaneously, the hash calculation module of the trusted central node performs a deterministic SHA-256 hash operation on the global gradient aggregation result generated by the decryption of this update cycle, i.e., the covering parameter, to generate a characteristic hash digest value unique to this update cycle.

[0059] The high-precision physical time acquisition module obtains the current trusted central physical time tag, accurate to the millisecond, from a local Network Time Protocol (NTP) time server or a GPS synchronized clock. Next, the cross-domain consortium blockchain interface module is invoked. This module concatenates the newly generated feature hash digest value with the trusted central physical time tag, and submits the concatenated result as the transaction content to a pre-configured cross-domain consortium blockchain network. This blockchain is maintained by multiple mutually trusted institutions, ensuring the immutability and traceability of the data on the chain. Consensus nodes in the network, using algorithms such as PBFT or Raft, verify the transaction, package it, and add it to a new block.

[0060] After a transaction is successfully recorded on the blockchain, the blockchain network sends a global smart contract beacon containing information such as the block generation timestamp, block hash, and Merkle root to the trusted central node through its tamper-proof mainnet link. This beacon is a data packet generated and broadcast by the blockchain network, containing an authoritative consensus record of this update operation. Its core value lies in the block generation timestamp guaranteed by the consensus of the entire network.

[0061] For example, following the gradient calculation and adaptive encryption module, assume that 10 edge nodes have completed gradient calculation and uploaded their respective encrypted gradient ciphertext blocks. The trusted central node receives these 10 ciphertext blocks. First, it verifies the ciphertext preimage in each ciphertext block. Then, it extracts the homomorphic encrypted portion, i.e., the 10... Perform additive aggregation of ciphertext fields, where the subscripts... Using the edge node numbers and the homomorphic addition property of Paillier encryption, the aggregated ciphertext is obtained. .

[0062] For 10 fundamental characteristic gradient flows subjected to noise perturbation Perform element-wise averaging. Assume the global gradient obtained after averaging and decryption is... Trust Center Perform a SHA-256 calculation to obtain the characteristic hash digest value of this update cycle, for example, "a3b8…7f21". Obtain the current time from the local clock server as "2023-10-26T10:30:05.123Z" as the trusted center's physical time stamp. The system concatenates these two strings into "a3b8…7f212023-10-26T10:30:05.123Z". Submit this string to the consortium blockchain via the blockchain API. The consortium blockchain's consensus nodes package a new block, for example, with block height #15789.

[0063] The block was ultimately confirmed with the timestamp "2023-10-26T10:30:08.456Z". The blockchain network then returned a transaction receipt to the trusted center, which contained the block's generation timestamp. This receipt data serves as the global smart contract beacon. The trusted center node stores this beacon for use in subsequent steps.

[0064] In one embodiment of the present invention, the dynamic token generation and parsing module is used to perform the following steps: The serialization concatenation calculation is performed on the cross-modal shared latent space feature vectors that reside in the local cache pool for a long time and the block generation time identifier; the direct concatenation is completed to generate the basic string for fusion operation; the basic string for fusion operation is fed into the one-way hash obfuscation algorithm conversion module to output a digest value with high variability; the digest value is independently encapsulated to generate a dynamic retrieval token as an anti-tampering verification identifier.

[0065] Specifically, after the preceding steps are completed, the system enters the dynamic token generation and parsing module. Local authorization modules deployed on edge workstations in various physical domains continuously monitor for new model parameters being distributed. Once a new round of periodic parameter matrices distributed by the trusted center has completed local iteration coverage, the module synchronously initiates its network monitoring process, intercepting and parsing the global smart contract beacon actively broadcast by the cross-domain consortium blockchain network via the P2P protocol. The parser is activated, performing structured parsing on the intercepted beacon data packets, filtering out irrelevant verification fields such as transaction signatures and node public keys that are not needed in this step. Subsequently, it extracts two key security elements from the reconstructed data stream: a block generation time stamp representing the global consensus time and global Merkle tree root node data representing the integrity of the block content. The block generation time stamp, parsed from the global smart contract beacon, is a precise timestamp of block creation determined by the blockchain consensus mechanism, typically a Unix timestamp with millisecond accuracy.

[0066] This stripping operation ensures that the target temporal features rely on the global consensus of the consortium blockchain network, thereby guaranteeing the anti-forgery and anti-tampering characteristics of the time base. Next, the authorization module retrieves the cross-modal shared latent space feature vector previously generated in the multimodal data acquisition and feature mapping module from the locally resident cache pool, and performs a mixing operation with the block generation time stamp just stripped from the beacon. This operation first serializes the feature vector and timestamp, which have different dimensions and data types, into binary byte streams. After serialization, the system performs a direct concatenation calculation on these two byte streams to generate a fusion operation base string that combines local spatial features and global time snapshot attributes. This fusion operation base string is the direct output of the feature string and time string concatenation operation, serving as a temporary unstructured binary data string.

[0067] Finally, the fusion operation base string is fed as the sole input into a transformation module configured with a one-way hashing algorithm. This module employs standard cryptographic hash algorithms such as SHA-256 or SHA-512. After the fusion operation base string is fed into this module, a fixed-length digest value with an avalanche effect is output. This digest value is then combined with the fusion operation base string to generate a dynamic retrieval token used to replace the traditional static key. Here, the generation process of the dynamic retrieval token can be represented by the following formula:

[0068] In the formula, This represents the summary feature portion of the final generated dynamic retrieval token; This represents the cross-modal shared latent space feature vector retrieved from the local cache; This indicates the block generation time identifier parsed from the beacon; The function converts physical quantities, such as floating-point vectors or time quantities, into bit strings; The function also operates on a bit string and outputs a dimensionless digest value; It is a concatenation operator.

[0069] For example, an edge workstation completes a model parameter update and intercepts a global smart contract beacon generated in the homomorphic aggregation and blockchain beacon generation module. The block generation time identifier parsed from the beacon is a Unix millisecond timestamp with a value of 1698316208456. The workstation retrieves a 256-dimensional cross-modal shared latent space feature vector generated for a specific event in the multimodal data acquisition and feature mapping module from its local cache, with values ​​[0.85, -0.21, ..., 0.55]. The system then begins generating tokens.

[0070] First, the 256-dimensional 32-bit floating-point vector [0.85, -0.21, ..., 0.55] is serialized into a 1024-byte binary stream. Serialize the 64-bit long integer Unix timestamp 1698316208456 into an 8-byte binary stream. The system will store 1024 bytes. With 8 bytes Serialization and concatenation are performed to generate a 1032-byte base string for fusion operations. This base string is then input into the SHA-512 algorithm module, which calculates and outputs a 64-byte, 512-bit hexadecimal digest value, such as "e8a1c3d9…b7f2". This digest value is packaged and encapsulated with the aforementioned 1032-byte base string to form the final dynamic retrieval token.

[0071] In one embodiment of the present invention, the knowledge graph connects the adjudication and construction modules, and is used to perform the following steps: The serialization concatenation calculation is performed on the cross-modal shared latent space feature vectors that reside in the local cache pool for a long time and the block generation time identifier; the direct concatenation is completed to generate the basic string for fusion operation; the basic string for fusion operation is fed into the one-way hash obfuscation algorithm conversion module to output a digest value with high variability; the digest value is independently encapsulated to generate a dynamic retrieval token as an anti-tampering verification identifier.

[0072] Specifically, after the dynamic retrieval token generated by the dynamic token generation and parsing module is successfully generated at the local edge workstation, the system enters the knowledge graph connection adjudication and construction module to build a secure and reliable knowledge graph network. When any edge device initiates a logical query connection request to establish a new cross-node physical association mapping in the global multimodal evidence chain knowledge graph network, such as associating a locally discovered video event with a network record in another node's log, the logic query interface deployed on the knowledge graph database computing architecture, such as the underlying native graph database Neo4j, activates its embedded high-speed connection request adjudication engine. This engine is a software module located on the database query processing path, used to perform permission and validity checks before executing the query.

[0073] This engine acts as a security checkpoint, interrupting the normal query process and compelling the requesting device to invoke and upload the dynamic retrieval token generated in its dynamic token generation and parsing module for the associated event. This token serves as the sole access lock for verifying the logical query boundary of the graph. Upon receiving the token, the adjudication engine does not immediately authorize but instead initiates a cross-verification process.

[0074] On the one hand, the original collection timestamp in the graph evidence chain is read. On the other hand, the collection timestamp serves as the physical timing information of hardware collection associated with the target event, which is reported and stored by the edge layer node that initiated the request.

[0075] On the other hand, digest features are extracted from the submitted dynamic retrieval tokens. After verifying the data's integrity through hash comparison, a locally pre-stored hash collision verification mechanism is executed to reconstruct the definitive block generation time identifier. Specifically, hash collision verification involves the adjudication engine generating a series of candidate block generation time identifiers within a reasonable physical time window centered on the original collection timestamp, such as a few seconds before or after. The locally pre-stored cross-modal shared latent space feature vector is then concatenated and hashed with each candidate time identifier. When the hash value obtained in a calculation perfectly matches the digest features in the token, the corresponding candidate time identifier is confirmed as the genuine block generation time identifier. This process leverages the limited guessability of timestamps for verification, completely eliminating the security vulnerability of using ciphertext to reverse-engineer features, i.e., the serialized data of the block generation time. This process is cryptographically feasible.

[0076] Subsequently, the adjudication engine calculates the deviation scale range between the two time tokens, i.e., the absolute value of the time difference between them. Based on this deviation value, the spatiotemporal decay penalty function is invoked, using the physical time lag error reflected by token decoding as the independent variable, to calculate and output an exponentially decreasing spatiotemporal decay penalty weight. This spatiotemporal decay penalty weight can be defined by the following formula:

[0077] In the formula, The spatiotemporal decay penalty weight is a value between 0 and 1, used to quantify the credibility of a connection request in the time dimension; It is the difference between the physical timing of hardware acquisition and the block generation time identifier, i.e., the physical time lag error, and its physical unit is limited to seconds. It is the spacetime penalty decay coefficient, and its unit is . This is used to control the rate at which the weight decreases as the time difference increases. The settings are based on the actual application scenarios' requirements for time synchronization accuracy. If the requirements are extremely stringent, A larger value should be chosen, such as 0.5. To achieve rapid penalty; if a certain tolerance is allowed, then a smaller value is taken, such as 0.1. .

[0078] When the local hardware timing matches the block consensus timing, the weight is 1; as the time lag error increases, the weight rapidly decays towards 0. Finally, the adjudication engine uses the negative feedback property of the spatiotemporal decay penalty weight to make a threshold judgment on the weight. If the calculated weight falls below a preset lower bound of 0.1, the connection request will be judged as a false replay connection operation, and the adjudication engine will not only reject the connection but also mark the request source as high-risk and restrict its subsequent operations. Conversely, for those entity node groups that successfully match the time sequence verification preimage, that is, those whose calculated spatiotemporal decay penalty weight is higher than the lower bound, the adjudication engine determines that their connection request has extremely high evidentiary value and grants them write access to the knowledge relationship logical connection, allowing them to create new entity nodes or relationship edges in the graph database.

[0079] Among them, the lower limit of the security boundary is the threshold for determining whether the connection is valid. The setting is based on balancing the security and availability of the system. For example, assuming that the normal network latency and clock synchronization error are within 5 seconds, the weight corresponding to the 5-second error can be calculated as the threshold based on the attenuation function.

[0080] By aggregating a large number of entity nodes that have undergone rigorous verification and have been authorized for writing, and their associated relationships, the system ultimately overlays, merges, and splices together to generate a secure, reliable, and time-consistent multimodal evidence chain knowledge graph network.

[0081] For example, edge device A attempts to associate its video observation node N1 with device B's log event node N2. Device A generates and uploads a dynamic retrieval token for this event, containing the fusion operation base string and the digest value "e8a1c3d9…b7f2". The underlying adjudication engine of the graph receives this token. It performs a hash check on the token's integrity, and upon confirmation, initiates reverse computation. The adjudication engine directly retrieves the cross-modal shared latent space feature vector [0.85, -0.21, ..., 0.55] bound to the request initiator node N1 through the graph index, and serializes it into 1024 bytes. .

[0082] The adjudication engine extracts the digest features carried in the token. Since the physical timing of hardware acquisition reported by device A is known to be 1698316210456, the engine generates candidate timestamps in millisecond increments within a 2-second time window before and after this time, and sequentially compares them with the feature serialization stream. The process involves concatenation and hash calculation. When the candidate timestamp 1698316208456 is encountered, the calculated hash value matches the "e8a1c3d9..." in the token, successfully restoring and confirming the block generation time identifier as 1698316208456. Simultaneously, the hardware acquisition physical timing sequence of the video observation result N1 reported by device A is read as timestamp 1698316210456. The engine calculates the physical time lag error. ms, or 2 s. Assuming an attenuation coefficient... Then calculate the spatiotemporal decay penalty weight. .

[0083] The adjudication engine compares this weight with the set lower bound of 0.1. Since 0.8187 > 0.1, the connection request is deemed valid. The adjudication engine then grants device A permission to create a new edge in the graph database pointing from node N1 to node N2, with a potential association and a weight attribute of 0.8187, thereby solidifying this time-verified connection into the multimodal evidence chain knowledge graph network. In one embodiment of the present invention, the knowledge graph connects the adjudication and construction modules, and is used to perform the following steps: The system acquires the physical timing sequence of hardware acquisition reported by the edge layer nodes of the graph; extracts the digest value from the received dynamic retrieval token; performs serialization and concatenation based on the pre-stored cross-modal shared latent space feature vector and the candidate block generation time identifiers traversed within a reasonable time window, and puts them into a one-way hash obfuscation algorithm conversion module to perform hash calculation; when the calculated hash value collides and matches with the digest value, the successfully matched candidate block generation time identifier is confirmed as the definitive block generation time identifier to be calculated and restored; cross-verifies the deviation scale range between the physical timing sequence of hardware acquisition and the calculated and restored block generation time identifier; calculates and outputs an exponentially downward spatiotemporal decay penalty weight based on the physical time lag error reflected by the decoding and comparison of the above time identifiers.

[0084] Specifically, after the knowledge graph connection adjudication and construction module builds and generates the basic multimodal evidence chain knowledge graph network, in order to address the evidence representation degradation problem that may be caused by differential privacy noise applied in previous steps and to finally confirm a complete evidence chain, the system enters the evidence chain compensation and traceability output module. This step is first triggered by a global task scheduler, which activates a bypass-deployed, edge-cloud collaborative teacher-student network compensation module based on an edge-cloud collaborative architecture. The workflow of this module is as follows: First, it scans all entity nodes in the multimodal evidence chain knowledge graph network and identifies those severely incomplete retrieval feature nodes that were heavily affected by differential noise interference due to low mutual information entropy during the generation process, resulting in unclear confidence or feature representation.

[0085] For each such node, the compensation module begins to execute the knowledge distillation feedback compensation algorithm, which is a model compression and optimization technique designed to transfer knowledge from the "teacher" model to the "student" model. In this embodiment, it is used for feature repair.

[0086] Specifically, a large "teacher" model, deployed on a cloud server and pre-trained using a globally preset cross-modal high-quality anchor feature set, processes anchor data semantically similar to the currently missing feature nodes, generating high-quality "soft labels"—logic values—as reference standards. This feature set is a calibration dataset constructed by domain experts, containing noise-free, highly correlated multimodal data samples, used to train the teacher model and guide the repair process.

[0087] Meanwhile, a small "student" model deployed at the edge device outputs its data through a compensation correction network. This network learns to reverse the effects of noise by minimizing the KL divergence loss between its output and the teacher model's soft labels, as well as the cross-entropy loss with the true labels. In this process, the loss function of the compensation correction network... It can be represented as:

[0088] In the formula, Therefore, a combined loss function is used; The current input to the student model (i.e., the compensation network) is the heavily incomplete retrieval feature that needs to be repaired, such as in the example below. ; The current input to the teacher model, i.e., the pre-extracted data and... High-quality anchor feature data with semantic similarity; The actual label corresponding to this anchor point data; The student model addresses incomplete features. The predicted output; It is a teacher model for clean anchor data Temperature coefficient The smoothed soft label output serves as a reference standard for repair; The student model addresses incomplete features. The corrected output is subjected to the same temperature coefficient Smoothed soft labels; It is the standard cross-entropy loss; Let KL be the divergence, used to measure the difference between two probability distributions; where... A mathematical separator used to measure the difference between two distributions.

[0089] These are weighting coefficients that balance the two types of losses, typically set between 0.1 and 0.3, to emphasize learning the generalization knowledge of the teacher model. This is because the problem involves heavily incomplete retrieval features, having lost a significant amount of complete source domain information. If the value is set too high, such as greater than 0.3, the student network will overfit the distorted labels of the incomplete samples, which will amplify the noise errors. Limiting it to a low value range of 0.1 to 0.3 will allow the network to mainly rely on the KL divergence of 0.7-0.9, which accounts for a large proportion, to passively learn the soft label manifold distribution of the 'teacher network' on the high-dimensional net feature plane, thereby maximizing the reverse reshaping of features contaminated by differential noise. Temperature is a hyperparameter in the distillation process, typically set to a value greater than 1, such as 2 to 5, to smooth the probability distribution and enable the student model to learn more information about the similarity between categories. If the temperature coefficient is too low, such as approaching 1, the soft label distribution is too sharp and cannot reflect the subtle topological relationships between different modal features; if it is too high, such as greater than 5, the label distribution tends to be absolutely uniform, losing the guiding significance of the error gradient. Selecting a range of 2 to 5 can optimally amplify the dark knowledge hidden in non-target classes through nonlinear mapping, guiding incomplete features to project onto the correct logical space manifold.

[0090] In this way, the heavily incomplete retrieval features are reshaped and reprojected in the feature space, so that their feature vectors are migrated from a low signal-to-noise ratio state to an optimized state that approximates a high-dimensional net feature plane. This is a theoretical feature space manifold that represents an ideal data distribution that is not affected by privacy noise.

[0091] After all the feature vectors of the incomplete nodes in the graph have been restored through precision compensation, these high-dimensional edge representation vectors that have been restored through precision compensation are designated as the starting key core pointers for the global network search. The system then starts the graph traversal engine and performs a breadth-first cross-comparison and multi-level leap-style judicial fact tracing along the paths in the multimodal evidence chain knowledge graph network that are connected to the boundary of judicially valid approval of the knowledge graph connection adjudication and building module, i.e., the spatiotemporal decay penalty weights stored on the connecting edges are higher than the lower limit of the safety boundary. Here, the boundary of judicially valid approval refers to the edges in the knowledge graph that represent real and reliable associations through the temporal verification of the knowledge graph connection adjudication and building module.

[0092] This traversal algorithm systematically explores all valid connection paths starting from the initial core pointer until a preset search depth is reached or no more valid connections can be found. The data integration module summarizes and aggregates each independent event activity clue stream with chain-like mutual verification attributes, which is tracked and integrated by the graph traversal engine and spans police and business awareness nodes across different geographical locations. The above traversal and integration process splices distributed independent evidence fragments into a spatiotemporally continuous cross-domain security evidence chain. This security evidence chain is finally formatted into a structured data report, such as a knowledge graph or time-series structure, and outputs to the outside world, presenting the complete logical clues of the cross-domain event.

[0093] For example, following the previous steps, the system discovers the feature vector of node N1 in the multimodal evidence chain knowledge graph network. Its confidence level is low. The system excitation compensation module repairs N1. A "running" video of the same type as N1 is selected from the high-quality anchor point feature set, i.e., the input. The teacher model processes the video and outputs soft tags. N1 That is, input After passing through the compensation network, the algorithm is continuously adjusted under the guidance of the loss function, and finally outputs the corrected, high-dimensional edge representation vector after accuracy compensation. Its distribution is closer to the output of the teacher model.

[0094] Subsequently, the system uses N1. Starting from N1, a breadth-first search is performed on the multimodal evidence chain knowledge graph network. The searcher first starts from N1 and finds an edge E1 leading to N2, checking its stored spatiotemporal decay penalty weight, which is 0.8187. Since this value is greater than 0.1, the searcher records the path N1->N2. Next, starting from N2, an edge E2 pointing to N3 is found, with a weight of 0.95, which is also greater than 0.1, so the path N1->N2->N3 is recorded. Assuming there are no more valid connections starting from N3, the search terminates, and this path is considered a complete event activity thread flow.

[0095] Finally, all such clue flows found are summarized, such as N1->N2->N3 mentioned above, and formatted into a report that shows the logical chain from "video evidence from device A" to "network logs from device B" and then to "follow-up video tracking from device C". This report is the final output of the factual cross-domain security evidence chain.

[0096] Each of the modules can be implemented in whole or in part through software, hardware, or a combination thereof. It supports hardware embedded in or independent of the processor in the computer device, and also supports software stored in the memory of the computer device, so that the processor can call and execute the operations corresponding to each of the above modules.

[0097] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A multimodal data cross-domain secure evidence chain construction system based on federated learning, characterized in that, include: The multimodal data acquisition and feature mapping module acquires multimodal physical sensing flow, extracts high-dimensional temporal feature matrix from preset feature encoding layer, performs cross-modal feature distance compression by calculating cross-head alignment correlation, and generates cross-modal shared latent space feature vector and cross-modal mutual information entropy feature value. The gradient calculation and adaptive encryption module performs backpropagation operations based on local business model parameters to extract basic feature gradient streams and cross-modal decision gradient streams. It imports cross-modal mutual information entropy feature values ​​to adjust the noise variance of the basic feature gradient streams and improves the encryption bit length barrier level of the cross-modal decision gradient streams, and aggregates them to generate encrypted gradient ciphertext blocks. The homomorphic aggregation and blockchain beacon generation module aggregates encrypted gradient ciphertext blocks to a preset trusted central node to perform homomorphic ciphertext feature aggregation, strips and decrypts to generate covering parameters, extracts feature hash digest values ​​and trusted central physical timestamps and inputs them into the cross-domain consortium blockchain network to generate a global smart contract beacon. The dynamic token generation and parsing module intercepts the global smart contract beacon and extracts the block generation time identifier. It performs serialization and concatenation calculations on the cross-modal shared latent space feature vector and the block generation time identifier to generate a fusion operation base string. Then, it generates a digest feature through a one-way hashing and obfuscation algorithm and independently encapsulates it to generate a dynamic retrieval token. The knowledge graph connection adjudication and construction module receives graph connection requests carrying dynamic retrieval tokens, extracts summary features from the dynamic retrieval tokens, performs hash collision verification based on pre-stored cross-modal shared latent space feature vectors and candidate block generation time identifiers traversed within a preset time window, matches and restores the block generation time identifiers, cross-verifies the preset hardware acquisition physical time sequence and block generation time identifiers to calculate the spatiotemporal decay penalty weight, cuts connection operations with weights lower than the preset security limit, and generates a multimodal evidence chain knowledge graph network. The evidence chain compensation and traceability output module uses a preset cross-modal high-quality anchor point feature set to guide the incomplete retrieval features to be reshaped into edge high-dimensional representation vectors. Using the edge high-dimensional representation vectors as core pointers, cross-comparison is performed along the multimodal evidence chain knowledge graph network to generate a cross-domain secure evidence chain.

2. The multimodal data cross-domain secure evidence chain construction system based on federated learning according to claim 1, characterized in that, Acquiring multimodal physical sensing flow includes the following steps: Activate distributed IoT edge computing nodes; Simultaneously acquire continuous time-series video images and structured logs of the communication network at the edge coverage site; Continuous video time-series images are merged and integrated with structured logs from communication networks to construct a multimodal physical sensing stream.

3. The multimodal data cross-domain secure evidence chain construction system based on federated learning according to claim 1, characterized in that, Generating cross-modal shared latent space feature vectors and cross-modal mutual information entropy feature values ​​includes the following steps: Constructing multi-head alignment correlations for high-dimensional temporal feature matrices; Calculate the local information overlap and crossover degree between multiple feature distribution databases; The heterogeneous signal stream is subjected to cross-modal feature distance compression, and the output is a cross-modal shared latent space feature vector with a unified feature dimension and a cross-modal mutual information entropy feature value.

4. The multimodal data cross-domain secure evidence chain construction system based on federated learning according to claim 1, characterized in that, Importing cross-modal mutual information entropy feature values ​​to adjust the noise variance of the basic feature gradient flow and improving the encryption bit length barrier level of the cross-modal decision gradient flow includes the following steps: Import cross-modal mutual information entropy feature values ​​as reverse compensation control factors and inject them into the underlying differential privacy generation module; Based on the changing trend of multi-physical flow coupling, noise variance is applied to the Laplace distribution of the basic feature gradient flow in real time. The redundant computing power released by reducing noise addition operations is extracted and sent to the improved homomorphic cryptographic processor, which forcibly increases the encryption bit length barrier level when processing cross-modal decision gradient flow.

5. The multimodal data cross-domain secure evidence chain construction system based on federated learning according to claim 1, characterized in that, The aggregation process generates encrypted gradient ciphertext blocks, including the following steps: The cross-modal mutual information entropy eigenvalues ​​are converted into ciphertext verification preimages through irreversible mathematical folding. The basic feature gradient flow after adjusting the noise variance, the cross-modal decision gradient flow after increasing the encryption bit length barrier level, and the ciphertext verification preimage are encapsulated. Aggregate and encapsulate the output encrypted gradient ciphertext block.

6. The multimodal data cross-domain secure evidence chain construction system based on federated learning according to claim 1, characterized in that, Extracting the feature hash digest value and the trusted center's physical time stamp, and inputting it into the cross-domain consortium blockchain network to generate a global smart contract beacon includes the following steps: Extract the feature hash digest value and the trusted center physical time tag generated in this update cycle; The concatenated feature hash digest value and the trusted center physical time tag are entered into the secure data ledger of the cross-domain consortium blockchain network to complete the verification consensus packaging; A global smart contract beacon containing a verifiable block generation timestamp is transmitted back via a tamper-proof mainnet link.

7. The multimodal data cross-domain secure evidence chain construction system based on federated learning according to claim 1, characterized in that, Intercepting the global smart contract beacon and extracting the block generation time identifier includes the following steps: The coverage of the parameter matrix iteration of each physical domain edge workstation in the access system in a new cycle is detected; Intercepting and dismantling global smart contract beacons actively broadcast by cross-domain consortium blockchain networks; The application system-level parser filters out irrelevant verification words and forcibly removes the block generation time stamp and global Merkle tree root node data from the reconstructed data stream.

8. The multimodal data cross-domain secure evidence chain construction system based on federated learning according to claim 1, characterized in that, The cross-modal shared latent space feature vector and the block generation time identifier are serialized and concatenated to generate a fusion operation base string, which is then used to generate a digest feature through a one-way hashing and obfuscation algorithm. This digest is then independently encapsulated to generate a dynamic retrieval token, including the following steps: Perform serialization concatenation calculation on the cross-modal shared latent space feature vectors that reside in the local cache pool for a long time and the block generation time identifier; Complete the direct concatenation to generate the basic string for fusion operation; The fusion operation base string is fed into the one-way hashing and obfuscation algorithm conversion module to output a digest value with highly variable characteristics; The digest value is encapsulated independently to generate a dynamic retrieval token that serves as an anti-tampering verification identifier.

9. The multimodal data cross-domain secure evidence chain construction system based on federated learning according to claim 1, characterized in that, Cross-validation of the preset hardware acquisition physical timing and block generation time identifier to calculate the spatiotemporal decay penalty weight includes the following steps: Obtain the hardware acquisition physical timing sequence reported by the edge layer nodes of the graph; Extract the digest value from the received dynamic retrieval token; Based on the pre-stored cross-modal shared latent space feature vector and the candidate block generation time identifier traversed within a reasonable time window, serialization and splicing are performed, and then fed into the one-way hash obfuscation algorithm conversion module to perform hash calculation; When the hash value of the calculated output collides and matches with the digest value, the successfully matched candidate block generation time identifier is confirmed as the block generation time identifier that has been solved and restored. Cross-verify the deviation scale range between the physical timing of hardware acquisition and the block generation time identifier reconstructed by calculation; The physical time lag error reflected by the above time stamp decoding comparison is calculated and output as an exponentially decreasing spatiotemporal decay penalty weight.

10. The multimodal data cross-domain secure evidence chain construction system based on federated learning according to claim 1, characterized in that, The process of reshaping incomplete retrieval features into high-dimensional edge representation vectors using a pre-defined cross-modal high-quality anchor feature set includes the following steps: Stimulate the edge-cloud collaborative teacher-student network compensation module guided and driven by a preset cross-modal high-quality anchor point feature set; Incomplete retrieval features that have suffered from confidence degradation due to differential noise interference in edge device execution layer localization; The incomplete retrieval features are guided to overcome the signal-to-noise ratio abyss and be re-projected onto the high-dimensional net feature plane to generate edge high-dimensional representation vectors.

Images