A dynamic perception credibility evaluation system and method for cross-modal federated learning

Abstract

The application relates to the technical field of federated learning, and discloses a dynamic perception credibility evaluation system and method for cross-modal federated learning, wherein the dynamic perception credibility evaluation method for cross-modal federated learning comprises the following steps: acquiring local multi-modal features and constructing a time-varying conditional dependence matrix to generate shared statistical information; analyzing cross-node dependence relationships and constructing a federated causal graph to output an inter-modal effect separation result; compressing local model parameters and extracting minimum sufficient statistics to output model contribution representation; calculating time-varying weights of node credibility to output dynamic node contribution degrees; performing adaptive federated aggregation and generating global model credibility evaluation to output a dynamically perceived global model; and the application provides a more reliable and more accurate model evaluation and aggregation mechanism for a distributed enterprise federated learning system, and effectively improves the overall performance and stability of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of federated learning, more particularly, it relates to a dynamic perception credibility evaluation system and method for cross-modal federated learning. BACKGROUND

[0002] In a distributed enterprise federated learning system, each branch holds heterogeneous modal data (including structured business data, text customer feedback, image product data, etc.), and needs to collaboratively train a global model while protecting data privacy. However, there are three key technical problems in the federated environment:

[0003] 1. Inconsistent data modalities at each node: the types and formats of data possessed by different nodes differ, making it difficult to directly integrate information from different modalities for global modeling;

[0004] 2. Local model quality varies significantly: due to uneven data quality, quantity, and distribution, the quality of local models trained by each node varies significantly;

[0005] 3. Time-varying causal dependence relationships between nodes are difficult to capture: the dependence relationships between nodes in the federated learning environment change dynamically over time, and traditional static aggregation methods cannot accurately capture this dynamic and causal relationship.

[0006] These problems result in an inability to accurately assess the contribution of each node to the global model and effectively evaluate the credibility of the global model, thereby affecting the overall performance and reliability of the federated learning system. SUMMARY

[0007] The present application provides a dynamic perception credibility evaluation system and method for cross-modal federated learning, solving the technical problem of being unable to accurately assess the contribution of each node and the credibility of the global model in related technologies.

[0008] The present application provides a dynamic perception credibility evaluation method for cross-modal federated learning, comprising the following steps:

[0009] Obtain local multi-modal features and construct a time-varying conditional dependence matrix to generate shared statistical information;

[0010] Analyze cross-node dependence relationships and construct a federated causal graph to output inter-modal effect separation results;

[0011] Compress local model parameters and extract minimal sufficient statistics to output model contribution representation;

[0012] Calculate time-varying weights of node credibility to output dynamic node contribution;

[0013] Perform adaptive federated aggregation and generate global model credibility evaluation to output a dynamically perceived global model;

[0014] The analysis of the cross-node dependency relationship adopts a variational causal decoupler, which separates direct effects within a mode and indirect effects between modes through latent variables, and generates a federated causal graph.

[0015] Further, the acquisition of the local multi-modal features and the construction of the time-varying conditional dependency matrix comprise:

[0016] Feature extraction is performed on the locally held multi-modal data to generate feature representations of each mode;

[0017] Based on the extracted features, a local time-varying conditional dependency matrix is calculated to represent the conditional dependency strength between features at different time points;

[0018] Statistical information is shared using a secure multi-party computation protocol, and a differential privacy mechanism is applied to protect the original statistical information.

[0019] Further, the compression of the local model parameters and the extraction of the minimum sufficient statistics comprise:

[0020] Based on the information bottleneck principle, the model parameters trained locally are compressed;

[0021] A variational approximation method is used to optimize the information bottleneck objective function;

[0022] The parameters of different modes are compressed using a mode-specific encoder network.

[0023] Further, the model parameter compression method comprises the following units:

[0024] Parameter encoding unit: using a variational autoencoder structure to compress high-dimensional model parameters into low-dimensional representations;

[0025] Information preservation unit: ensuring that the compressed representation retains task-related information through contrastive learning;

[0026] Privacy enhancement unit: adding noise interference to prevent reconstruction of the original data.

[0027] Further, the time-varying weight of the node credibility comprises:

[0028] Input the statistics extracted by each node into a dynamic Bayesian network;

[0029] Combine the performance of the historical training rounds to calculate the node credibility;

[0030] Based on the trained dynamic Bayesian network, calculate the time-varying weight for each node;

[0031] Apply an anomaly detection mechanism to identify potential malicious nodes.

[0032] Further, the dynamic Bayesian network is composed of the following component units:

[0033] Time series dependency modeling unit: capturing time series patterns of node statistics;

[0034] Credibility evaluation unit: calculating node credibility based on multi-dimensional features;

[0035] Anomaly detection unit: identifying abnormal patterns in time series;

[0036] Weight generation unit: converting credibility scores into aggregation weights.

[0037] Further, the execution of adaptive federated aggregation and generation of global model credibility evaluation includes:

[0038] Adjusting aggregation weights based on node credibility and contribution;

[0039] Considering the causal relationship between modalities, adjusting aggregation weights according to the federated causal graph;

[0040] After completing adaptive aggregation, evaluate the credibility of the global model by considering the contribution quality, causal consistency and time stability.

[0041] Further, the adaptive federated aggregation method is composed of the following component units:

[0042] Weight adjustment unit: calculating effective weights based on node credibility and abnormal state;

[0043] Modality-aware unit: processing specific aggregation strategies for different modal data;

[0044] Time consistency unit: ensuring the time continuity of the aggregation process;

[0045] Credibility evaluation unit: comprehensive calculation of credibility indicators of global model.

[0046] Further, the variational causal decoupler adopts a time-aware attention mechanism, enabling the model to adaptively focus on key dependency relationships in different time periods.

[0047] The present application provides a dynamic perception credibility evaluation system for cross-modal federated learning, which is used to execute the aforementioned dynamic perception credibility evaluation method for cross-modal federated learning, comprising:

[0048] Feature extraction module, for acquiring local multi-modal features and constructing time-varying conditional dependency matrix;

[0049] Causal decoupling module, for analyzing cross-node dependency relationships and constructing a federated causal graph;

[0050] a parameter compression module configured to compress local model parameters and extract minimum sufficient statistics;

[0051] a credibility evaluation module configured to calculate time-varying weights of node credibility;

[0052] a federated aggregation module configured to perform adaptive federated aggregation and generate global model credibility evaluation;

[0053] The system realizes dynamic perception credibility evaluation of the global model of the cross-modal federated learning by dynamically adjusting the node weights and the modal weights.

[0054] The present application has the beneficial effects that the global model credibility evaluation problem caused by multi-modal data heterogeneity, node quality difference and time-varying dependence in the federated learning environment is solved, a more reliable and more accurate model evaluation and aggregation mechanism is provided for the distributed enterprise federated learning system, and the overall performance and stability of the system are effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a flowchart of the dynamic perception credibility evaluation method of the cross-modal federated learning of the present application;

[0056] Figure 2 is a line graph of the time-varying credibility weight change of each bank node in the federated learning training round, which shows the time-varying credibility weight change in the application example of the financial institution alliance risk assessment system in the patent;

[0057] Figure 3 is a comparison graph of model performance and privacy protection level under different privacy budgets of the present application, which shows the balance relationship between model performance and privacy protection level under different privacy budget (ε) settings in the patent;

[0058] Figure 4 is a performance comparison bar chart of different aggregation methods when there is an abnormal node of the present application, which compares the performance difference between the traditional average aggregation method and the method of the present application when there are different proportions of abnormal nodes;

[0059] Figure 5 is a multi-modal data flow and model contribution san chart of the present application, which shows how the multi-modal data of different nodes flows to the global model in the patent;

[0060] Figure 6 is a feature correlation change relationship graph before and after the variational causal decoupling of the present application, which shows the change of the correlation between different features before and after the processing of the variational causal decoupler in the patent;

[0061] Figure 7This is a scatter plot showing the relationship between the compression rate and information retention of the information bottleneck framework of this invention, illustrating the relationship between the compression rate and the retention of task-related information under different hyperparameters β. Detailed Implementation

[0062] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0063] Example 1

[0064] This implementation provides a dynamic perceptual credibility evaluation system and method for cross-modal federated learning, such as... Figure 1 As shown, the method includes the following steps:

[0065] Step 1: Obtain local multimodal features and construct a time-varying conditional dependency matrix to generate shared statistical information;

[0066] In this implementation, each federated node first extracts features from its locally held multimodal data to generate feature representations for each modality. Specifically, for each federated node i, its local data can be represented as a set of multiple modalities:

[0067]

[0068] Where M represents the number of modes. The data for node i for each mode. Apply the appropriate feature extractor Modal features are obtained. .

[0069] For structured data, the feature extractor receives the original structured data as input and outputs standardized and dimensionality-reduced feature vectors.

[0070] For text data, the text encoder receives the raw text as input and outputs a semantic feature vector.

[0071] For image data, a convolutional neural network takes the original image as input and outputs a visual feature vector.

[0072] Before feature extraction, appropriate data preprocessing is performed for different modalities:

[0073] The structured data is normalized to unify the numerical characteristics of different dimensions into the [0,1] interval;

[0074] One-hot Encoding is applied to categorical features to convert them into numerical representations.

[0075] Text data is tokenized, stop words are removed, and word stems are extracted.

[0076] Image data is resized, pixel values are normalized, and data augmentation is applied.

[0077] These preprocessing steps ensure the stability and comparability of different modal data during feature extraction.

[0078] Based on the extracted features, each node computes a local time-varying conditional dependence matrix The local time-varying conditional dependence matrix represents the conditional dependence strength between features at different time points t.

[0079]

[0080] where represents the conditional mutual information between feature j and feature k at time point t.

[0081] Each node shares statistical information, not raw data or features, using secure multi-party computation protocols. Specifically, node i generates a set of statistical summaries including but not limited to inter-modal mutual information estimates, time-varying dependence strength, and other statistical quantities.

[0082] where the differential privacy mechanism receives the original statistical information as input and outputs privacy-protected statistical information with added noise, denoted as

[0083]

[0084] where is the noise generated according to the pre-set privacy budget ε.

[0085] Step 2: Analyze cross-node dependence relationships and construct a federated causal graph, outputting the inter-modal effect separation results.

[0086] After the central server collects statistical information from all nodes , it analyzes the dependence relationships between nodes using a variational causal decoupler. The variational causal decoupler is based on the principle of variational inference, which separates the causal relationships between different modalities by learning latent variable representations.

[0087] Before performing the variational causal decoupling, the central server standardizes the collected statistics to eliminate the dimensional differences between different nodes and different modalities. Specifically, the Z-score standardization is applied to each statistical indicator to convert the individual statistics into a standard normal distribution with a mean of 0 and a standard deviation of 1, ensuring the comparability of statistics of different sources and types and avoiding the excessive influence of some indicators with large magnitudes in the decoupling process.

[0088] Specifically, the objective function of the variational causal decoupler is:

[0089]

[0090] where, represents the expectation under the latent variable distribution output by the encoder network , is the encoder network, is the decoder network, is the latent variable , is the KL divergence term, is the regularization term for promoting causal decoupling (used to quantify the independence between dimensions in the latent variable , calculated as the square norm of the sum of the off-diagonal elements of the latent variable covariance matrix), is the trade-off parameter.

[0091] Further, the variational causal decoupler is composed of the following component units:

[0092] Encoder network: receives the node statistics and maps them to the latent variable space;

[0093] Causal decoupling module: decomposes the latent variables and identifies direct and indirect causal relationships;

[0094] Decoder network: reconstructs the decoupled latent variables back to the original space;

[0095] Causal graph construction unit: constructs the federated causal graph based on the decoupling results.

[0096] The data transmission relationship between the component units is as follows:

[0097] First, the encoder network encodes the statistics into latent variables ;

[0098] Then, the causal decoupling module processes to obtain decomposed latent variables ;

[0099] Next, the decoder network reconstructs Reconstructing back to the original space gives ;

[0100] Finally, the causal graph construction unit generates a federated causal graph atlas based on

[0101] By optimizing the above objective function, the variational causal decoupler can decompose the cross-node dependency into:

[0102] Intra-modal direct effect: direct causal relationship between features within the same modality;

[0103] Inter-modal indirect effect: indirect influence between features of different modalities through intermediate variables.

[0104] Based on this decomposition, the central server generates a federated causal graph atlas:

[0105]

[0106] where V is the node set (representing features of different modalities), E is the edge set (representing causal relationships between features), and W is the edge weight (representing causal strength).

[0107] In the embodiments of the present application, in order to more accurately capture the time-varying characteristics, the variational causal decoupler adopts a time-aware attention mechanism, so that the model can adaptively focus on key dependency relationships in different time periods. The attention weight is calculated as follows:

[0108]

[0109] where, is the attention score function (a function that calculates the similarity of statistical information at time points t and t', which is implemented as the normalized dot product of the statistical information at the two time points), and are the statistical information at time points t and t', respectively. The statistical information vector of the candidate time point has the same dimension as , and all candidate time points are traversed to calculate the softmax denominator.

[0110] Step 3: compress local model parameters and extract minimal sufficient statistics, output model contribution representation;

[0111] Based on the information bottleneck principle, each federated node compresses the model parameters trained locally and extracts the minimal sufficient statistics that contribute to the global model. The information bottleneck framework receives the local model parameters as input and outputs the compressed model representation, retaining key information valuable to the global model.

[0112] For the local model parameters of federated node i​ The optimization objective of the information bottleneck framework is:

[0113]

[0114] where, is the mutual information between the compressed representation and the original parameters is the mutual information between and the target task Y, is a hyperparameter that controls the degree of compression, is the probability encoding that generates the compressed representation given .

[0115] Further, the model parameter compression method in this embodiment includes the following units:

[0116] Parameter encoding unit: compress high-dimensional model parameters into low-dimensional representation using variational autoencoder structure;

[0117] Information preservation unit: ensure that the compressed representation retains task-related information through contrastive learning;

[0118] Privacy enhancement unit: add noise interference to prevent reconstruction of original data.

[0119] The data transmission process between these units is:

[0120] First, the parameter encoding unit receives the model parameters and outputs the preliminary compressed representation;

[0121] Then, the information preservation unit optimizes the compressed representation to ensure that key task information is retained;

[0122] Finally, the privacy enhancement unit adds an appropriate amount of noise to generate the final compressed statistics .

[0123] Since direct calculation of mutual information has high computational complexity in high-dimensional space, this embodiment uses variational approximation method to convert the above optimization objective into:

[0124]

[0125] where, is the parameter encoder, is the task predictor, is the prior distribution.

[0126] In this way, each node obtains the minimum sufficient statistics of local model contribution , and the minimum sufficient statistics of local model contribution has the following characteristics:​

[0127] Key information valuable for the global model is preserved;

[0128] Redundant information of the original parameters is minimized;

[0129] Direct leakage of the original data is avoided.

[0130] Wherein, in order to adapt to the characteristics of different modalities, the embodiment adopts a modal-specific encoder network for the parameters of different modalities:

[0131]

[0132] Wherein, is the modal m-specific encoder parameter, is the model parameter related to modal m in node i, represents the compressed representation extracted from by the encoder.

[0133] Step 4: Calculate the time-varying weight of node credibility, output dynamic node contribution degree;

[0134] The statistics extracted by each node are input into the dynamic Bayesian network, and the time-varying weight of node credibility is calculated combined with the performance of the historical training rounds. The dynamic Bayesian network receives the statistics extracted by the nodes and the historical performance data as input, and outputs the time-varying weight of node credibility.

[0135] Before inputting into the dynamic Bayesian network, the time series data is preprocessed:

[0136] Firstly, uniformization processing is performed on the timestamps to ensure the consistency of time intervals;

[0137] Then, minimum-maximum normalization is applied to different types of indicators to map the numerical values to the [0, 1] interval, so that different indicators can be compared on the same scale;

[0138] For missing values, forward filling or interpolation method based on time series context is used for completion;

[0139] Finally, sliding window technology is applied to extract time series features to capture short-term and long-term dependencies.

[0140] In the embodiment, the structure of the dynamic Bayesian network is defined as:

[0141]

[0142] Wherein represents the initial network structure, represents the time transition network structure.

[0143] Furthermore, the dynamic Bayesian network in this embodiment is composed of the following components:

[0144] Temporal dependency modeling unit: captures time-series patterns of node statistics;

[0145] Credibility assessment unit: Calculates node credibility based on multi-dimensional features;

[0146] Anomaly detection unit: identifies anomalous patterns in time series data;

[0147] Weight generation unit: Converts credibility scores into aggregate weights.

[0148] The data transfer relationships between these units are as follows:

[0149] First, the time-series dependency modeling unit processes the time-series data of the nodes and extracts time-varying features;

[0150] Then, the credibility assessment unit combines these features and historical performance to calculate a credibility score;

[0151] Next, the anomaly detection unit marks potential anomalous behaviors;

[0152] Finally, the weight generation unit integrates all the information to generate the final time-varying weights.

[0153] For each federated node i, its credibility at time t is calculated using the following formula:

[0154]

[0155] in, It is the reliability index of node i at time t. It is the statistic provided by node i at time t. is the historical performance metric of node i, and k is the size of the time window.

[0156] To calculate the conditional probabilities mentioned above, the variational inference method takes the observed sequence as input and outputs parameters of an approximate posterior distribution, optimizing the following lower bound of evidence (ELBO):

[0157]

[0158] in, Indicates from time 1 to The sequence of latent variables, This represents the observed data sequence.

[0159] Based on the trained dynamic Bayesian network, time-varying weights are calculated for each node:

[0160]

[0161] wherein, is a temperature parameter, controlling the smoothness of the weight distribution, is the total number of nodes.

[0162] In the embodiments of the present application, the anomaly detection mechanism receives the historical trustworthiness sequence of the nodes as input and outputs an anomaly state indicator of the federated node at the current time point, which is used to identify potential malicious nodes.

[0163] Step 5: Perform adaptive federated aggregation and generate global model trustworthiness evaluation, output dynamic perception global model;

[0164] Based on the decoupled causal weights and node trustworthiness obtained in the preceding steps, perform adaptive federated aggregation to generate a global model trustworthiness evaluation with dynamic perception capability. The core of adaptive aggregation is to dynamically adjust the aggregation weights according to the trustworthiness and contribution of the nodes.

[0165] Before performing adaptive aggregation, the model parameters of each node are preprocessed:

[0166] First, apply norm normalization to the parameter tensor to ensure the comparability of parameter amplitudes between different nodes;

[0167] Then, detect and prune abnormal value parameters to avoid the adverse effects of extreme values on the aggregation results;

[0168] Finally, adjust the parameter weights for sparsity, and use corresponding sparsity thresholds for different modal parameters to balance the expression ability and complexity of the model.

[0169] These preprocessing steps ensure the fair contribution of different nodes and different modal parameters in the aggregation process.

[0170] Further, the adaptive federated aggregation method in the present embodiment is composed of the following component units:

[0171] Weight adjustment unit: calculate effective weights based on node trustworthiness and anomaly state;

[0172] Modality perception unit: handle specific aggregation strategies for different modal data;

[0173] Temporal consistency unit: ensure the time continuity of the aggregation process;

[0174] Trustworthiness evaluation unit: comprehensively calculate the trustworthiness index of the global model.

[0175] The data transmission relationship between these units is:

[0176] First, the weight adjustment unit receives the node trustworthiness and anomaly state to calculate the aggregation weights;

[0177] Then, the modality awareness unit adjusts the aggregation strategy of each modality according to the modality characteristics;

[0178] Next, the temporal consistency unit integrates historical information to ensure smooth aggregation;

[0179] Finally, the credibility assessment unit calculates the global model credibility based on the aggregation results.

[0180] For global model parameters The adaptive aggregation formula is:

[0181]

[0182] Where, is the credibility weight of node i at time t, is the anomaly indicator of node i at time t, is the local model parameter of node i at time t.

[0183] For different modalities of features and model parameters, the aggregation process considers the causal relationship between modalities, and adjusts the aggregation weight according to the federal causal graph generated by the variational causal decoupler:

[0184]

[0185] Where, is the aggregation weight of node i at time t for modality m, is the set of nodes connected to modality m in the causal graph, is the causal strength between modality m and node is the set of all modalities.

[0186] After adaptive aggregation, the global model credibility assessment receives the aggregated model parameters and each node contribution index as input, and outputs the credibility score of the global model, which considers factors such as contribution quality, causal consistency and temporal stability.

[0187] In the embodiments of the present application, further, the modality adaptive unit receives the performance indicators of each modality in the current task as input, and outputs the importance weight of each modality, which is used to dynamically adjust the contribution proportion of different modalities in the global model aggregation process.

[0188] The following is an example of an application of the present application, as shown in Figures 2-7 The implementation process is as follows:

[0189] Local multi-modal feature extraction and dependency matrix construction:

[0190] ​Each bank node first extracts features from its local data. Take institution A as an example, its structured transaction data goes through the following processing:

[0191] Table 1: Example of structured data feature extraction results for institution A

[0192]

[0193] Institution B applies the BERT text encoder to extract features from the text data:

[0194] Table 2: Example of text data feature extraction results for institution B

[0195]

[0196] Based on the extracted features, institution A calculates the local time-varying conditional dependence matrix, and the following shows part of the results at two different time points (t1 and t2):

[0197] Table 3: Part of the conditional dependence matrix of institution A at two time points

[0198]

[0199] Each institution generates a differentially private protected statistical information summary and shares it to the central server. The added noise level is determined based on a privacy budget of ε = 3.0, ensuring that sensitive information is adequately protected.

[0200] Variational Causal Decoupling and Federated Causal Graph Construction:

[0201] After the central server collects the statistical information from each bank node, it applies the variational causal decoupler to analyze cross-node dependency relationships and construct a federated causal graph.

[0202] During the decoupling process, the encoder of the variational causal decoupler maps high-dimensional statistical information to a 10-dimensional latent variable space, and then the causal decoupling module identifies direct and indirect causal relationships. Finally, the central server generates a federated causal graph containing 57 nodes and 142 edges.

[0203] Table 4: Time-varying credibility weights of each bank node in the 10 rounds of federated learning training

[0204]

[0205] It can be observed that as the number of training rounds increases, the weight of institution E decreases significantly, because the system detects abnormalities in its data quality. While the weights of the other four institutions gradually stabilize, indicating that the system recognizes their stable contribution to the global model.

[0206] The modal adaptive unit in the system receives the performance indicators of each modality in the current task, and dynamically adjusts the contribution proportion of different modalities. Specifically, the structured data parameters use weighted average, the text features use weighted attention mechanism, and the image features use weighted principal component analysis. This modal adaptive mechanism ensures that the global model can fully utilize the advantages of each modality.

[0207] It can be understood that the data preprocessing methods known to those skilled in the art include data cleaning, data conversion, and data reduction, wherein the data conversion includes type conversion and normalization and standardization. Although the dimensions and types of data are ignored in the description of the foregoing embodiments, data preprocessing is a technical knowledge known to those skilled in the art and a prerequisite for data processing, and therefore the known data preprocessing steps are not independently described in the foregoing content.

[0208] The above describes the embodiments of the present application, but the embodiments are not limited to the specific embodiments described above, and the specific embodiments described above are only illustrative but not restrictive. Those skilled in the art can also make more forms of equivalent embodiments under the inspiration of the embodiments, which are all within the protection of the embodiments.

Claims

1. A dynamic perceptual credibility evaluation method for cross-modal federated learning, characterized in that, Includes the following steps: Acquire local multimodal features and construct a time-varying conditional dependency matrix to generate shared statistical information; The process of acquiring local multimodal features and constructing a time-varying conditional dependency matrix includes: Feature extraction is performed on locally held multimodal data to generate feature representations for each modality; Based on the extracted features, the local time-varying conditional dependency matrix is ​​calculated to characterize the strength of conditional dependency between features at different time points. Statistical information is shared using a secure multi-party computation protocol, and the original statistical information is protected using a differential privacy mechanism. Analyze cross-node dependencies and construct a federated causal graph, outputting the results of intermodal effect separation; Compress the local model parameters and extract the minimum sufficient statistics to produce a model contribution representation; Calculate the time-varying weights of node credibility and output the dynamic node contribution. Perform adaptive federated aggregation and generate a global model credibility assessment, outputting a dynamically aware global model; Among them, the variational causal decoupling device is used to analyze cross-node dependencies. The variational causal decoupling device separates intramodal direct effects and intermodal indirect effects through latent variable representation, and generates a federated causal map.

2. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 1, characterized in that, The process of compressing local model parameters and extracting the minimum sufficient statistics includes: Based on the principle of information bottleneck, the parameters of locally trained models are compressed. Optimize the information bottleneck objective function using variational approximation methods; The parameters of different modes are compressed using a mode-specific encoder network.

3. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 2, characterized in that, The model parameter compression method includes the following units: Parameter encoding unit: Uses a variational autoencoder structure to compress high-dimensional model parameters into a low-dimensional representation; Information retention unit: Through contrastive learning, it ensures that the compressed representation retains task-related information; Privacy Enhancement Unit: Adds noise interference to prevent reconstruction of the original data.

4. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 1, characterized in that, The time-varying weights for the reliability of the computing nodes include: The statistics extracted from each node are input into the dynamic Bayesian network; Calculate node reliability based on performance across historical training rounds; Based on the trained dynamic Bayesian network, time-varying weights are calculated for each node; Anomaly detection mechanisms are used to identify potential malicious nodes.

5. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 4, characterized in that, The dynamic Bayesian network is composed of the following components: Temporal dependency modeling unit: captures time-series patterns of node statistics; Credibility assessment unit: Calculates node credibility based on multi-dimensional features; Anomaly detection unit: identifies anomalous patterns in time series data; Weight generation unit: Converts credibility scores into aggregate weights.

6. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 1, characterized in that, The process of performing adaptive federated aggregation and generating a global model credibility assessment includes: The aggregation weight is dynamically adjusted based on node credibility and contribution. Considering the causal relationships between modalities, the aggregation weights are adjusted based on the federated causal graph; After adaptive aggregation is completed, the credibility of the global model is evaluated by comprehensively considering contribution quality, causal consistency, and temporal stability.

7. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 6, characterized in that, The adaptive federated aggregation method consists of the following components: Weight adjustment unit: Calculates effective weights based on node credibility and abnormal states; Modality-aware unit: A specific aggregation strategy for processing data with different modalities; Temporal consistency unit: ensures the temporal continuity of the aggregation process; Credibility assessment unit: comprehensively calculates the credibility index of the global model.

8. The dynamic perceptual credibility evaluation method for cross-modal federated learning according to claim 1, characterized in that, The variational causal decoupler employs a time-aware attention mechanism, enabling the model to adaptively focus on key dependencies within different time periods.

9. A dynamic perceptual credibility evaluation system for cross-modal federated learning, used to execute the dynamic perceptual credibility evaluation method for cross-modal federated learning as described in any one of claims 1-8, characterized in that, include: The feature extraction module is used to acquire local multimodal features and construct a time-varying conditional dependency matrix; The causal decoupling module is used to analyze cross-node dependencies and construct a federated causal graph; The parameter compression module is used to compress local model parameters and extract the minimum sufficient statistics; The credibility assessment module is used to calculate the time-varying weights of node credibility. The federated aggregation module is used to perform adaptive federated aggregation and generate a global model credibility assessment. The system achieves dynamic perception and credibility evaluation of the global model of cross-modal federated learning by dynamically adjusting node weights and modal weights. The process of acquiring local multimodal features and constructing a time-varying conditional dependency matrix includes: Feature extraction is performed on locally held multimodal data to generate feature representations for each modality; Based on the extracted features, the local time-varying conditional dependency matrix is ​​calculated to characterize the strength of conditional dependency between features at different time points. Statistical information is shared using a secure multi-party computation protocol, and the original statistical information is protected using a differential privacy mechanism.

Images