Dynamic perception credibility evaluation system and method for cross-modal federated learning

Abstract

The invention relates to the technical field of federated learning, and discloses a dynamic perception credibility evaluation system and method for cross-modal federated learning, and the method comprises the steps: obtaining local multi-modal features, constructing a time-varying condition dependence matrix, and generating shared statistical information; analyzing a cross-node dependency relationship, constructing a federal causal map, and outputting an inter-modal effect separation result; local model parameters are compressed, the minimum sufficient statistics are extracted, and model contribution characterization is output; calculating a time-varying weight of the node credibility, and outputting a dynamic node contribution degree; executing adaptive federal aggregation, generating global model credibility evaluation, and outputting a dynamically perceived global model; according to the invention, a more reliable and more accurate model evaluation and aggregation mechanism is provided for the distributed enterprise federated learning system, and the overall performance and stability of the system are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of federated learning technology, and more specifically, to a dynamic perception credibility evaluation system and method for cross-modal federated learning. Background Technology

[0002] In distributed enterprise federated learning systems, branch offices hold heterogeneous modal data (including structured business data, textual customer feedback, image product data, etc.) and need to collaboratively train a global model while protecting data privacy. However, three key technical challenges exist in a federated environment: 1. Inconsistent data modalities among nodes: Different nodes possess different data types and formats, making it difficult to directly integrate information from different modalities for global modeling; 2. Inconsistent local model quality: Due to the imbalance in data quality, quantity, and distribution, there are significant differences in the quality of local models trained at different nodes; 3. Time-varying causal dependencies between nodes are difficult to capture: In a federated learning environment, the dependencies between nodes change dynamically over time, and traditional static aggregation methods cannot accurately capture such dynamic causal relationships.

[0003] These issues make it impossible to accurately assess the contribution of each node to the global model, nor can they effectively evaluate the credibility of the global model, thus affecting the overall performance and reliability of the federated learning system. Summary of the Invention

[0004] This invention provides a dynamic perception credibility evaluation system and method for cross-modal federated learning, which solves the technical problem in related technologies that cannot accurately evaluate the contribution of each node and the credibility of the global model.

[0005] This invention provides a dynamic perceptual credibility evaluation method for cross-modal federated learning, comprising the following steps: Acquire local multimodal features and construct a time-varying conditional dependency matrix to generate shared statistical information; Analyze cross-node dependencies and construct a federated causal graph, outputting the results of intermodal effect separation; Compress the local model parameters and extract the minimum sufficient statistics to produce a model contribution representation; Calculate the time-varying weights of node credibility and output the dynamic node contribution. Perform adaptive federated aggregation and generate a global model credibility assessment, outputting a dynamically aware global model; Among them, the variational causal decoupling device is used to analyze cross-node dependencies. The variational causal decoupling device separates intramodal direct effects and intermodal indirect effects through latent variable representation, and generates a federated causal map.

[0006] Furthermore, the step of acquiring local multimodal features and constructing a time-varying conditional dependency matrix includes: Feature extraction is performed on locally held multimodal data to generate feature representations for each modality; Based on the extracted features, the local time-varying conditional dependency matrix is ​​calculated to characterize the strength of conditional dependency between features at different time points. Statistical information is shared using a secure multi-party computation protocol, and the original statistical information is protected using a differential privacy mechanism.

[0007] Furthermore, the compression of local model parameters and extraction of minimum sufficient statistics includes: Based on the principle of information bottleneck, the parameters of locally trained models are compressed. Optimize the information bottleneck objective function using variational approximation methods; The parameters of different modes are compressed using a mode-specific encoder network.

[0008] Furthermore, the model parameter compression method includes the following units: Parameter encoding unit: Uses a variational autoencoder structure to compress high-dimensional model parameters into a low-dimensional representation; Information retention unit: Through contrastive learning, it ensures that the compressed representation retains task-related information; Privacy Enhancement Unit: Adds noise interference to prevent reconstruction of the original data.

[0009] Furthermore, the time-varying weights for the credibility of the computing nodes include: The statistics extracted from each node are input into the dynamic Bayesian network; Calculate node reliability based on performance across historical training rounds; Based on the trained dynamic Bayesian network, time-varying weights are calculated for each node; Anomaly detection mechanisms are used to identify potential malicious nodes.

[0010] Furthermore, the dynamic Bayesian network is composed of the following components: Temporal dependency modeling unit: captures time-series patterns of node statistics; Credibility assessment unit: Calculates node credibility based on multi-dimensional features; Anomaly detection unit: identifies anomalous patterns in time series data; Weight generation unit: Converts credibility scores into aggregate weights.

[0011] Furthermore, the step of performing adaptive federated aggregation and generating a global model credibility assessment includes: The aggregation weight is dynamically adjusted based on node credibility and contribution. Considering the causal relationships between modalities, the aggregation weights are adjusted based on the federated causal graph; After adaptive aggregation is completed, the credibility of the global model is evaluated by comprehensively considering contribution quality, causal consistency, and temporal stability.

[0012] Furthermore, the adaptive federated aggregation method comprises the following components: Weight adjustment unit: Calculates effective weights based on node credibility and abnormal states; Modality-aware unit: A specific aggregation strategy for processing data with different modalities; Temporal consistency unit: ensures the temporal continuity of the aggregation process; Credibility assessment unit: comprehensively calculates the credibility index of the global model.

[0013] Furthermore, the variational causal decoupler employs a time-aware attention mechanism, enabling the model to adaptively focus on key dependencies within different time periods.

[0014] This invention provides a dynamic perceptual credibility evaluation system for cross-modal federated learning, used to execute the aforementioned dynamic perceptual credibility evaluation method for cross-modal federated learning, comprising: The feature extraction module is used to acquire local multimodal features and construct a time-varying conditional dependency matrix; The causal decoupling module is used to analyze cross-node dependencies and construct a federated causal graph; The parameter compression module is used to compress local model parameters and extract the minimum sufficient statistics; The credibility assessment module is used to calculate the time-varying weights of node credibility. The federated aggregation module is used to perform adaptive federated aggregation and generate a global model credibility assessment. The system achieves dynamic perception and credibility assessment of the global model of cross-modal federated learning by dynamically adjusting node weights and modal weights.

[0015] The beneficial effects of this invention are: it solves the problem of global model credibility evaluation caused by multimodal data heterogeneity, node quality differences and time-varying dependencies in federated learning environments, and provides a more reliable and accurate model evaluation and aggregation mechanism for distributed enterprise federated learning systems, effectively improving the overall performance and stability of the system. Attached Figure Description

[0016] Figure 1 This is a flowchart of the dynamic perception credibility evaluation method for cross-modal federated learning of the present invention; Figure 2This is a line graph showing the time-varying credibility weight changes of each bank node in the federated learning training rounds of this invention, illustrating the time-varying credibility weight changes in the application example of the financial institution consortium risk assessment system in the patent. Figure 3 This is a comparison chart of model performance and privacy protection level under different privacy budgets of the present invention, which shows the balance between model performance and privacy protection level under different privacy budget (ε) settings in the patent; Figure 4 This is a bar chart comparing the performance of different aggregation methods in the presence of abnormal nodes, comparing the performance differences between the traditional average aggregation method and the method of this patent under different proportions of abnormal nodes. Figure 5 This invention presents a Sankey diagram illustrating the multimodal data flow and model contribution of the present invention, demonstrating how multimodal data from different nodes flows to the global model in the patent. Figure 6 This is a diagram showing the changes in the correlation between features before and after the variational causal decoupling of the present invention, illustrating the changes in the correlation between different features before and after processing by the variational causal decoupling device in the patent. Figure 7 This is a scatter plot showing the relationship between the compression rate and information retention of the information bottleneck framework of this invention, illustrating the relationship between the compression rate and the retention of task-related information under different hyperparameters β. Detailed Implementation

[0017] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0018] Example 1 This implementation provides a dynamic perceptual credibility evaluation system and method for cross-modal federated learning, such as... Figure 1 As shown, the method includes the following steps: Step 1: Obtain local multimodal features and construct a time-varying conditional dependency matrix to generate shared statistical information; In this implementation, each federated node first extracts features from its locally held multimodal data to generate feature representations for each modality. Specifically, for each federated node i, its local data can be represented as a set of multiple modalities:

[0019] Where M represents the number of modes. The data for node i for each mode. Apply the appropriate feature extractor Modal features are obtained. .

[0020] For structured data, the feature extractor receives the original structured data as input and outputs standardized and dimensionality-reduced feature vectors. For text data, the text encoder receives the raw text as input and outputs a semantic feature vector. For image data, a convolutional neural network takes the original image as input and outputs a visual feature vector.

[0021] Before feature extraction, appropriate data preprocessing is performed for different modalities: The structured data is normalized to unify the numerical characteristics of different dimensions into the [0,1] interval; The categorical features are converted into numerical representations using one-hot encoding. Perform word segmentation, stop word removal, and stemming on text data; Image data is subjected to size standardization, pixel value normalization, and data augmentation.

[0022] These preprocessing steps ensure the stability and comparability of data from different modalities during feature extraction.

[0023] Based on the extracted features, each node calculates its local time-varying conditional dependency matrix. The local time-varying conditional dependency matrix characterizes the strength of conditional dependencies between features at different time points t:

[0024] in, This represents the conditional mutual information between feature j and feature k at time point t.

[0025] Each node shares statistical information, rather than raw data or features, using a secure multi-party computation protocol. Specifically, node i generates a set of statistical summaries. This includes, but is not limited to, statistics such as intermodal mutual information estimation and time-varying dependency strength.

[0026] The differential privacy mechanism receives raw statistical information as input and outputs privacy-preserving statistical information with added noise, as follows:

[0027] in, It is noise generated based on a preset privacy budget ε.

[0028] Step 2: Analyze cross-node dependencies and construct a federated causal graph, outputting the results of intermodal effect separation; The central server collects statistical information from all nodes. Then, a variational causal decoupler is used to analyze the dependencies between nodes. The variational causal decoupler is based on the principle of variational inference and separates the causal relationships between different modalities by learning the latent variable representation.

[0029] Before performing variational causal decoupling, the central server standardizes the collected statistical information to eliminate dimensional differences between different nodes and modalities. Specifically, Z-score standardization is applied to each statistical indicator, converting each statistic into a standard normal distribution with a mean of 0 and a standard deviation of 1. This ensures the comparability of statistical information from different sources and types, and avoids excessive influence from certain large-scale indicators during the decoupling process.

[0030] Specifically, the objective function of the variational causal decoupler is:

[0031] in, In the encoder network The expected value under the latent variable distribution of the output. It is an encoder network. It is a decoder network. It is a latent variable The prior distribution (usually the standard normal distribution) is taken. It is the KL divergence term. It is a regularization term that promotes causal decoupling (used to quantify latent variables). The independence between the dimensions is calculated as the square norm of the sum of the off-diagonal elements of the latent variable covariance matrix. It is a trade-off parameter.

[0032] Furthermore, the variational causal decoupler consists of the following components: Encoder network: Receives statistical information from each node and maps it to the latent variable space; Causal decoupling module: decomposes latent variables and identifies direct and indirect causal relationships; Decoder network: Reconstructs the decoupled latent variables back into the original space; Causal graph construction unit: Construct a federated causal graph based on the decoupling results.

[0033] The data transfer relationships between the constituent units are as follows: First, the encoder network will collect statistical information. Encoding as latent variables ; Then, the causal decoupling module processes... Obtain the latent variables after decomposition ; Next, the decoder network will Reconstructing back to the original space yields ; Finally, the causal graph building unit is based on Generate a federal causal graph.

[0034] By optimizing the above objective function, the variational causal decoupling algorithm can decompose cross-node dependencies into: Intramodal direct effects: direct causal relationships between features within the same mode; Intermodal indirect effects: the indirect influence between different modal characteristics through intermediate variables.

[0035] Based on this decomposition, the central server generates a federated causal graph:

[0036] Where V is the set of nodes (representing features of different modalities), E is the set of edges (representing causal relationships between features), and W is the edge weight (representing causal strength).

[0037] In this embodiment, to more accurately capture time-varying characteristics, the variational causal decoupler employs a time-aware attention mechanism, enabling the model to adaptively focus on key dependencies within different time periods. The attention weights are calculated as follows:

[0038] in, It is the attention scoring function (a function that calculates the statistical similarity between time points t and t', specifically implemented as the standardized dot product of the statistical information of the two time points). and These are the statistical information for time points t and t', respectively. Candidate time points for normalization The statistical information vector, and With the same dimension, iterate through all candidate time points to calculate the softmax denominator.

[0039] Step 3: Compress the local model parameters and extract the minimum sufficient statistics to produce a model contribution representation; Based on the information bottleneck principle, each federated node compresses the model parameters trained locally, extracting the minimum sufficient statistics that contribute to the global model. The information bottleneck framework receives local model parameters as input and outputs a compressed model representation, retaining key information valuable to the global model.

[0040] For the local model parameters of federated node i The optimization objective of the information bottleneck framework is:

[0041] in, It is a compressed representation and original parameters Mutual information between them yes Mutual information between the target task Y and the target task Y It is a hyperparameter that controls the degree of compression. Therefore Conditional generation of compressed representation The probability encoding.

[0042] Furthermore, the model parameter compression method in this embodiment includes the following units: Parameter encoding unit: Uses a variational autoencoder structure to compress high-dimensional model parameters into a low-dimensional representation; Information retention unit: Through contrastive learning, it ensures that the compressed representation retains task-related information; Privacy Enhancement Unit: Adds noise interference to prevent reconstruction of the original data.

[0043] The data transfer process between these units is as follows: First, the parameter encoding unit receives the model parameters. It also outputs a preliminary compressed representation; Then, the information retention unit optimizes the compressed representation to ensure that critical mission information is retained; Finally, the privacy enhancement unit adds an appropriate amount of noise to generate the final compressed statistics. .

[0044] Since directly calculating mutual information is computationally complex in high-dimensional spaces, this implementation method uses a variational approximation method to transform the above optimization objective into:

[0045] in, It is a parameter encoder. It is a task predictor. It is a prior distribution.

[0046] In this way, each node obtains the minimum sufficient statistics of its local model contribution. The minimum sufficient statistic for the contribution of the local model has the following properties: Key information valuable to the global model was preserved; It minimizes redundant information in the original parameters; This avoids the direct leakage of raw data.

[0047] In order to adapt to the characteristics of different modes, this embodiment uses a mode-specific encoder network for the parameters of different modes:

[0048] in, These are encoder parameters specific to mode m. These are the model parameters in node i that are related to mode m. Indicates that from the encoder Extracted compressed representation.

[0049] Step 4: Calculate the time-varying weight of node credibility and output the dynamic node contribution. The statistical measures extracted from each node are input into the dynamic Bayesian network, and the time-varying weights of node credibility are calculated by combining the performance data from previous training rounds. The dynamic Bayesian network receives the statistical measures extracted from the nodes and historical performance data as input, and outputs the time-varying weights of node credibility.

[0050] Before inputting the dynamic Bayesian network, the time series data is preprocessed: First, the timestamps are uniformized to ensure the consistency of time intervals; Then, minimum-maximum normalization is applied to different types of indicators to map the values ​​to the [0,1] interval, so that different indicators can be compared on the same scale. For missing values, forward padding or interpolation based on temporal context is used to complete them; Finally, the sliding window technique is applied to extract temporal features to capture short-term and long-term dependencies.

[0051] In this embodiment, the structure of the dynamic Bayesian network is defined as follows:

[0052] in This represents the initial network structure. This represents the time-transition network structure.

[0053] Furthermore, the dynamic Bayesian network in this embodiment is composed of the following components: Temporal dependency modeling unit: captures time-series patterns of node statistics; Credibility assessment unit: Calculates node credibility based on multi-dimensional features; Anomaly detection unit: identifies anomalous patterns in time series data; Weight generation unit: Converts credibility scores into aggregate weights.

[0054] The data transfer relationships between these units are as follows: First, the time-series dependency modeling unit processes the time-series data of the nodes and extracts time-varying features; Then, the credibility assessment unit combines these features and historical performance to calculate a credibility score; Next, the anomaly detection unit marks potential anomalous behaviors; Finally, the weight generation unit integrates all the information to generate the final time-varying weights.

[0055] For each federated node i, its credibility at time t is calculated using the following formula:

[0056] in, It is the reliability index of node i at time t. It is the statistic provided by node i at time t. is the historical performance metric of node i, and k is the size of the time window.

[0057] To calculate the conditional probabilities mentioned above, the variational inference method takes the observed sequence as input and outputs parameters of an approximate posterior distribution, optimizing the following lower bound of evidence (ELBO):

[0058] in, Indicates from time 1 to The sequence of latent variables, This represents the observed data sequence.

[0059] Based on the trained dynamic Bayesian network, time-varying weights are calculated for each node:

[0060] in, It is a temperature parameter that controls the smoothness of the weight distribution. It represents the total number of nodes.

[0061] In this embodiment, the anomaly detection mechanism receives the historical credibility sequence of nodes as input and outputs an anomaly status indicator of the federated nodes at the current time point to identify potential malicious nodes.

[0062] Step 5: Perform adaptive federated aggregation and generate a global model credibility assessment, outputting a dynamically aware global model; Based on the decoupled causal weights and node credibility obtained in the preceding steps, adaptive federated aggregation is performed to generate a global model credibility assessment with dynamic awareness. The core of adaptive aggregation is to dynamically adjust the aggregation weights according to the node's credibility and contribution.

[0063] Before performing adaptive aggregation, the model parameters of each node are preprocessed: First, norm normalization is applied to the parameter tensor to ensure the comparability of parameter amplitudes between different nodes; Then, outlier parameters are detected and pruned to avoid the adverse effects of extreme values ​​on the aggregation results; Finally, the sparsity of the parameter weights is adjusted, and corresponding sparsity thresholds are used for parameters of different modalities to balance the expressive power and complexity of the model.

[0064] These preprocessing steps ensure fair contributions from different nodes and modal parameters during the aggregation process.

[0065] Furthermore, the adaptive federated aggregation method in this embodiment consists of the following components: Weight adjustment unit: Calculates effective weights based on node credibility and abnormal states; Modality-aware unit: A specific aggregation strategy for processing data with different modalities; Temporal consistency unit: ensures the temporal continuity of the aggregation process; Credibility assessment unit: comprehensively calculates the credibility index of the global model.

[0066] The data transfer relationships between these units are as follows: First, the weight adjustment unit receives the node's credibility and abnormal status, and calculates the aggregate weight; Then, the modal sensing unit adjusts the aggregation strategy of each modality according to the modal characteristics; Next, the timing consistency unit integrates historical information to ensure smooth aggregation; Finally, the credibility assessment unit calculates the global model credibility based on the aggregation results.

[0067] For global model parameters Its adaptive aggregation formula is:

[0068] in, It is the credibility weight of node i at time t. It is an anomaly indicator for node i at time t. These are the local model parameters of node i at time t.

[0069] For different modalities and model parameters, the aggregation process considers the causal relationships between modalities and adjusts the aggregation weights based on the federated causal graph generated by the variational causal decoupler.

[0070] in, It is the aggregate weight of node i with respect to mode m at time t. It is the set of nodes in the causal graph that are connected to mode m. It is mode m and node The causal strength between them It is the set of all modalities.

[0071] After adaptive aggregation is completed, the global model credibility assessment takes the aggregated model parameters and the contribution indicators of each node as input and outputs the credibility score of the global model, which comprehensively considers factors such as contribution quality, causal consistency and time stability.

[0072] In this embodiment, the modality adaptation unit further receives the performance index of each modality in the current task as input and outputs the importance weight of each modality, which is used to dynamically adjust the contribution ratio of different modalities during the global model aggregation process.

[0073] The following is an example of an application of the present invention, such as Figure 2-7 As shown, the implementation process is as follows: Local multimodal feature extraction and dependency matrix construction: Each bank node first extracts features from its local data. Taking institution A as an example, its structured transaction data undergoes the following processing: Table 1: Example of feature extraction results for structured data of Organization A

[0074] Organization B applied the BERT text encoder to extract features from the text data: Table 2: Examples of Feature Extraction Results for Text Data from Organization B

[0075] Based on the extracted features, organization A calculated the local time-varying conditional dependency matrix. The following shows partial results at two different time points (t1 and t2): Table 3: Partial Example of the Conditional Dependency Matrix of Institution A at Two Time Points

[0076] Each organization generates a differentially privacy-preserving statistical summary and shares it with the central server. The added noise level is determined based on a privacy budget of ε=3.0 to ensure that sensitive information is adequately protected.

[0077] Variational causal decoupling and federated causal graph construction: After collecting statistical information from each bank node, the central server uses a variational causal decoupler to analyze cross-node dependencies and construct a federated causal graph.

[0078] During the decoupling process, the encoder of the variational causal decoupler maps high-dimensional statistical information to a 10-dimensional latent variable space, and then the causal decoupling module identifies direct and indirect causal relationships. Finally, the central server generates a federated causal graph containing 57 nodes and 142 edges.

[0079] Table 4: Time-varying credibility weights of each bank node in 10 rounds of federated learning training

[0080] It can be observed that as the number of training rounds increases, the weight of institution E decreases significantly, because the system detects anomalies in its data quality. Meanwhile, the weights of the other four institutions gradually stabilize, indicating that the system recognizes their stable contributions to the global model.

[0081] The modality adaptation unit in the system receives performance metrics of each modality in the current task and dynamically adjusts the contribution ratio of different modalities. Specifically, structured data parameters use weighted averaging, text features use a weighted attention mechanism, and image features use weighted principal component analysis. This modality adaptation mechanism ensures that the global model can fully utilize the advantages of each modality.

[0082] It is understood that data preprocessing methods known to those skilled in the art include data cleaning, data transformation, and data reduction. Data transformation includes type conversion and normalization and standardization. Although the dimensions and types of data were omitted in the description of the preceding embodiments, data preprocessing is a technical knowledge known to those skilled in the art and a prerequisite step in data processing. Therefore, the previously described well-known data preprocessing steps were not described independently.

[0083] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A dynamic perceptual credibility evaluation method for cross-modal federated learning, characterized in that, Includes the following steps: Acquire local multimodal features and construct a time-varying conditional dependency matrix to generate shared statistical information; Analyze cross-node dependencies and construct a federated causal graph, outputting the results of intermodal effect separation; Compress the local model parameters and extract the minimum sufficient statistics to produce a model contribution representation; Calculate the time-varying weights of node credibility and output the dynamic node contribution. Perform adaptive federated aggregation and generate a global model credibility assessment, outputting a dynamically aware global model; Among them, the variational causal decoupling device is used to analyze cross-node dependencies. The variational causal decoupling device separates intramodal direct effects and intermodal indirect effects through latent variable representation, and generates a federated causal map.

2. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 1, characterized in that, The process of acquiring local multimodal features and constructing a time-varying conditional dependency matrix includes: Feature extraction is performed on locally held multimodal data to generate feature representations for each modality; Based on the extracted features, the local time-varying conditional dependency matrix is ​​calculated to characterize the strength of conditional dependency between features at different time points. Statistical information is shared using a secure multi-party computation protocol, and the original statistical information is protected using a differential privacy mechanism.

3. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 1, characterized in that, The process of compressing local model parameters and extracting the minimum sufficient statistics includes: Based on the principle of information bottleneck, the parameters of locally trained models are compressed. Optimize the information bottleneck objective function using variational approximation methods; The parameters of different modes are compressed using a mode-specific encoder network.

4. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 3, characterized in that, The model parameter compression method includes the following units: Parameter encoding unit: Uses a variational autoencoder structure to compress high-dimensional model parameters into a low-dimensional representation; Information retention unit: Through contrastive learning, it ensures that the compressed representation retains task-related information; Privacy Enhancement Unit: Adds noise interference to prevent reconstruction of the original data.

5. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 1, characterized in that, The time-varying weights for the reliability of the computing nodes include: The statistics extracted from each node are input into the dynamic Bayesian network; Calculate node reliability based on performance across historical training rounds; Based on the trained dynamic Bayesian network, time-varying weights are calculated for each node; Anomaly detection mechanisms are used to identify potential malicious nodes.

6. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 5, characterized in that, The dynamic Bayesian network is composed of the following components: Temporal dependency modeling unit: captures time-series patterns of node statistics; Credibility assessment unit: Calculates node credibility based on multi-dimensional features; Anomaly detection unit: identifies anomalous patterns in time series data; Weight generation unit: Converts credibility scores into aggregate weights.

7. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 1, characterized in that, The process of performing adaptive federated aggregation and generating a global model credibility assessment includes: The aggregation weight is dynamically adjusted based on node credibility and contribution. Considering the causal relationships between modalities, the aggregation weights are adjusted based on the federated causal graph; After adaptive aggregation is completed, the credibility of the global model is evaluated by comprehensively considering contribution quality, causal consistency, and temporal stability.

8. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 7, characterized in that, The adaptive federated aggregation method consists of the following components: Weight adjustment unit: Calculates effective weights based on node credibility and abnormal states; Modality-aware unit: A specific aggregation strategy for processing data with different modalities; Temporal consistency unit: ensures the temporal continuity of the aggregation process; Credibility assessment unit: comprehensively calculates the credibility index of the global model.

9. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 1, characterized in that, The variational causal decoupler employs a time-aware attention mechanism, enabling the model to adaptively focus on key dependencies within different time periods.

10. A dynamic perceptual credibility evaluation system for cross-modal federated learning, used to execute a dynamic perceptual credibility evaluation method for cross-modal federated learning as described in any one of claims 1-9, characterized in that, include: The feature extraction module is used to acquire local multimodal features and construct a time-varying conditional dependency matrix; The causal decoupling module is used to analyze cross-node dependencies and construct a federated causal graph; The parameter compression module is used to compress local model parameters and extract the minimum sufficient statistics; The credibility assessment module is used to calculate the time-varying weights of node credibility. The federated aggregation module is used to perform adaptive federated aggregation and generate a global model credibility assessment. The system achieves dynamic perception and credibility assessment of the global model of cross-modal federated learning by dynamically adjusting node weights and modal weights.

Images