Financial deep counterfeiting detection and prevention system and method based on multi-modal large model

Abstract

The invention discloses a financial deep counterfeiting real-time detection and defense method and system based on a multi-modal large model, and the method comprises the steps: obtaining multi-modal data in a financial transaction scene, and carrying out the desensitization of an edge end; performing dynamic time sequence alignment on the multi-modal data, and calculating a synchronous error of lip motion and voice by adopting a dynamic time warping algorithm; inputting the features into a dynamic risk modeling layer, and generating dynamic risk features in combination with the updated risk feature library; analyzing the features through a double-flow GAN detector, and outputting a forgery probability; and a detection result is input into a compliance verification layer, the supervision file is analyzed through a legal BERT, a structured rule is generated, and real-time transaction interception and block chain log recording are executed. The method protects user privacy and data security, combines the risk feature library updated in real time, and has high flexibility and adaptability. According to the design of the double-flow GAN detector, image and video stream information is fully utilized, and the accuracy and reliability of detection are further improved.

Description

Technical Field

[0001] The present invention relates to the field of financial security technology, and in particular to a financial deep forgery detection and prevention system and method based on a multimodal large model. Background Art

[0002] The field of financial security technology faces an increasingly severe threat from deepfakes. With the rapid development of artificial intelligence (AI), deepfake technology is gradually moving from the laboratory to real-world applications, particularly in the fintech industry, where it poses unprecedented challenges to identity verification, transaction security, and data privacy. Deepfake technology utilizes large models to generate highly realistic multimedia content, such as video, audio, and images, which is sufficient to deceive ordinary users and even circumvent traditional identity verification systems. In the fintech sector, deepfakes are primarily used in intelligent customer service, risk assessment, customer service, and identity authentication, placing significant pressure on the security of financial institutions.

[0003] Traditional financial security and anti-counterfeiting methods are insufficiently equipped to withstand deepfakes. Traditional anti-counterfeiting systems often rely on a single modality, such as facial or voiceprint recognition, making them easily circumvented by targeted deepfake attacks. Furthermore, traditional anti-counterfeiting models have long iteration cycles, making them incapable of rapidly responding to emerging new counterfeiting methods. Therefore, financial institutions urgently need a technology that can effectively identify and defend against a wide range of deepfake attacks to ensure transaction security and user privacy.

[0004] The patent with publication number CN113889122B discloses a training method for a voiceprint recognition model, a voiceprint recognition method, and related equipment. Among them, the training method of the voiceprint recognition model specifically includes the following steps: inputting the voiceprint data into the model to be trained multiple times for iterative training until a voiceprint recognition model is obtained. The model to be trained is composed of L network layers, where L is an integer greater than 1. During the Nth iterative training, the following operations are performed: the voiceprint data is passed through the L network layers layer by layer for feature mapping, and finally a voiceprint vector is output. If the output value corresponding to the voiceprint vector meets the preset conditions, the model to be trained after the Nth iterative training is determined to be a voiceprint recognition model, where N is a positive integer. This method effectively improves the sample utilization rate and enhances the accuracy of the trained voiceprint recognition model.

[0005] In recent years, the application of multimodal large-scale models in the field of financial security and anti-counterfeiting has gradually gained attention. Multimodal large-scale models can combine multiple biometric features and information, such as face, voiceprint, and lip reading, for cross-validation, greatly improving the accuracy and robustness of anti-counterfeiting. In addition, multimodal large-scale models possess strong generalization and rapid iteration capabilities, enabling them to quickly adapt to the emergence of new counterfeiting methods. Therefore, financial deepfake detection and prevention systems and methods based on multimodal large-scale models have become a current research hotspot in the field of financial security technology and are of great significance for improving the security protection capabilities of financial institutions. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a financial deep fake detection and prevention system and method based on a multimodal large model, aiming to solve the user privacy and data security problems existing in the existing technology when processing multimodal data. By obtaining multimodal data and performing desensitization processing at the edge, the data processing efficiency and accuracy can be improved.

[0007] The object of the present invention is achieved through the following technical solutions: A real-time detection and defense method for financial deepfakes based on a multimodal large model, including the following: S1: Acquire multimodal data from financial transaction scenarios, including text instructions, facial images, and voice recordings, and perform edge desensitization on each modality. S2: Dynamically align the multimodal data using a dynamic time warping algorithm to align the speech and video timelines and calculate the synchronization error between lip movements and speech. S3: Input the time-aligned features into the dynamic risk modeling layer and generate dynamic risk features based on the real-time updated risk feature library. The risk feature library pulls the latest attack pattern features from the threat intelligence platform every X hours. S4: Analyze dynamic risk features through the deepfake detection layer's two-stream GAN detector, which includes a ResNet-50 for the image stream and a Transformer for the video stream, outputs forgery probabilities, and identifies cross-modal inconsistencies. S5: Input the detection results into the compliance verification layer, parse the regulatory documents through legal BERT and generate structured rules, perform real-time transaction interception and blockchain logging.

[0008] As a preferred embodiment, it also includes: S6: Update the detection model parameters based on the federated learning framework, use Paillier homomorphic encryption to aggregate gradients, and inject Gaussian noise to ensure differential privacy. S7: Calculate the comprehensive score of the detection results. If the capacity of the dynamic feature library exceeds the threshold, replace the historical features based on the score and trigger secondary authentication.

[0009] As a preferred method, the specific process of dynamic timing alignment in S2 includes: S21: Calibrate the clock of the multimodal data acquisition device through the PTP protocol to control the synchronization error within ±1ms; S22: Using a buffer to store data of each modality, and performing optical flow interpolation compensation for missing frames after a timeout; S23: Calculate the lip-speech synchronization error Δt. If Δt>50ms, it is determined to be a forgery attack.

[0010] As a preferred method, the training process of the two-stream GAN detector in S4 includes: S41: The generator generates fake videos and synchronized speech based on StyleGAN3, constraining the Fréchet Inception Distance between the generated samples and the real data to be ≤15; S42: The detector uses a focal loss function to jointly optimize the classification loss and reconstruction loss; S43: Alternately train the generator and detector until the detector's recognition accuracy for the latest generated samples is ≥ 95% and remains stable for 3 rounds.

[0011] As a preferred method, the process of generating structured rules in S5 includes: S51: Using Legal BERT to parse regulatory documents and extract IF-THEN logic rules; S52: Mapping the transaction amount, IP risk score, and voiceprint matching degree into a risk level; S53: If the risk score exceeds the threshold, the transaction is automatically intercepted and a blockchain audit log is generated.

[0012] As a preferred method, the parameter update process of federated learning in S6 includes: S61: Each participant trains the detection model locally, performs Paillier encryption on the gradient parameters, and then uploads them; S62: The aggregation server decrypts the gradient and updates the global model; S63: Compressing the global model into MobileViT via knowledge distillation.

[0013] As a preferred method, the calculation method of the comprehensive score in S7 includes: S71: Calculate the total confidence score based on the predicted entropy of the current test result and the historical average entropy; S72: taking the ratio of the total confidence score to the preset threshold as the comprehensive score; S73: If the score is ≥ the lowest score in the dynamic feature library, replace the corresponding feature and update the model.

[0014] As a preferred approach, the rules for cross-modal contradiction analysis include: Rule 1: If the voiceprint matching degree is less than 0.8 and the IP address is overseas, a high-risk alert is triggered; Rule 2: If the transaction amount is greater than 3 times the user's historical average, manual review is mandatory.

[0015] A real-time financial deepfake detection and defense system based on a multimodal large model, including: Multimodal input module: used to collect text instructions, facial images, and voice recordings, and perform desensitization processing at the edge node; Timing alignment module: aligns multimodal data using a dynamic time warping algorithm and calculates lip-speech synchronization errors; Dynamic risk modeling module: contains a real-time updated risk feature library that stores attack pattern features and frequency domain noise fingerprints; Deepfake Detection Module: This includes a dual-stream GAN detector and a cross-modal contradiction analysis engine, outputting forgery probability and risk score; Compliance Verification Module: Analyzes financial regulatory rules and generates structured interception strategies, connecting to the blockchain audit system; Federated learning module: uses Paillier encryption to aggregate gradients and protects data security through differential privacy; Real-time defense module: Dynamically updates the signature database based on comprehensive scores, triggering SMS authentication or transaction interception.

[0016] As a preferred embodiment, the dual-stream GAN detector includes: Image stream: Extract spatial artifact features based on ResNet-50; Video stream: Transformer-based encoding of inter-frame temporal features; Fusion layer: weights multimodal features through a cross-modal attention mechanism.

[0017] The present invention has at least the following beneficial effects: The present invention effectively protects user privacy and data security by acquiring and processing multimodal data, including text instructions, facial images and voice recordings, and performing desensitization processing at the edge, while improving the efficiency and accuracy of data processing. Secondly, a dynamic time warping algorithm is used to align the time sequence of multimodal data, calculate the synchronization error between lip movement and voice, and accurately identify the timing inconsistency problem in counterfeit behavior, thereby improving the detection accuracy. Combined with the real-time updated risk feature library, the dynamic risk modeling layer can generate dynamic risk features that reflect the latest attack patterns, so that the detection method has a high degree of flexibility and adaptability, and can respond to new counterfeit attacks in a timely manner. The design of the dual-stream GAN detector makes full use of the information of image and video streams. Through the synergy of image stream ResNet-50 and video stream Transformer, it can accurately output the probability of counterfeiting and identify cross-modal contradictions, further improving the accuracy and reliability of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To reveal the technical details of the embodiments of the present invention, the following is a brief introduction to the drawings involved in the embodiments. It should be emphasized that these drawings only illustrate several embodiments of the present invention and should not be considered as defining the scope of the invention. Those skilled in the art can deduce other relevant drawings based on these drawings without engaging in creative work.

[0019] Figure 1 Schematic diagram of detection and defense methods; Figure 2 This is a schematic diagram of the detection and defense system structure; Figure 3 Schematic diagram of the two-stream GAN training process. DETAILED DESCRIPTION

[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0021] In the following, embodiments of the present disclosure are described in detail with the aid of accompanying drawings. However, please be aware that the present disclosure is not limited to the specific forms shown herein. Rather, it should be understood to encompass various variations, equivalents, and / or alternatives to the embodiments of the present disclosure. In describing the drawings, the same reference numerals will be used to indicate similar components.

[0022] In this disclosure, terms are used to illustrate specific embodiments and do not constitute limitations of this disclosure. In this context, the use of the singular also encompasses the plural, unless the text clearly indicates otherwise. In the process of explanation, it should be understood that terms such as "including" or "having" are intended to indicate the presence of a feature, quantity, step, operation, structural component, part, or combination thereof, and do not preclude the possibility or addition of one or more other features, quantities, steps, operations, structural components, parts, or combinations thereof.

[0023] It should be understood that while the following description provides extensive specific details intended to facilitate a comprehensive understanding of the example embodiments, those skilled in the art will appreciate that the example embodiments can be implemented without these specific details. For example, systems may be presented in block diagram form to avoid excessive detail that would obscure the clarity of the examples. In other cases, unnecessary details regarding well-known processes, structures, and techniques may be omitted to maintain clarity of the examples.

[0024] like Figure 1 As shown in the figure, a real-time detection and defense method for financial deep fakes based on a multimodal large model includes the following: S1: Acquire multimodal data from financial transaction scenarios, including text instructions, facial images, and voice recordings, and perform edge desensitization on each modality. S2: Dynamically align the multimodal data using the dynamic time warping (DTW) algorithm to align the speech and video time axes and calculate the synchronization error between lip movement and speech. S3: Input the time-aligned features into the dynamic risk modeling layer and generate dynamic risk features based on the real-time updated risk feature library. The risk feature library pulls the latest attack pattern features from the threat intelligence platform every X hours (X is a positive number, such as 2). S4: Analyze dynamic risk features through the deepfake detection layer's two-stream GAN detector, which includes a ResNet-50 for the image stream and a Transformer for the video stream, outputs forgery probabilities, and identifies cross-modal inconsistencies. S5: Input the detection results into the compliance verification layer, parse the regulatory documents through legal BERT and generate structured rules, perform real-time transaction interception and blockchain logging.

[0025] This embodiment integrates multimodal data analysis and dynamic risk modeling to accurately identify and effectively prevent potential forgeries in financial transaction scenarios. This approach not only improves the security of financial transactions but also effectively reduces the economic losses and compliance risks caused by deepfake attacks. The compliance verification layer uses Legal BERT to parse regulatory documents and generate structured rules, enabling real-time interception of transaction behavior and blockchain logging, effectively preventing financial risks and ensuring the security and compliance of financial transactions.

[0026] In a preferred embodiment, it also includes: S6: Update the detection model parameters based on the federated learning framework, use Paillier homomorphic encryption to aggregate gradients, and inject Gaussian noise to ensure differential privacy. S7: Calculate the comprehensive score of the detection results. If the capacity of the dynamic feature library exceeds the threshold, replace the historical features based on the score and trigger secondary authentication.

[0027] The federated learning model update mechanism uses a two-stream neural network as its core detection model (ResNet-50 is used to extract facial artifacts in the image stream, and Transformer is used to analyze temporal features in the video stream). After each financial institution trains the model locally using its own data, it converts the model parameter gradients (i.e., the parameter adjustment direction) into ciphertext and uploads it using Paillier homomorphic encryption. This encryption algorithm allows the central server to directly perform mathematical operations (such as weighted averaging) on the encrypted gradients, aggregating parameters from multiple parties without decryption, fundamentally eliminating the risk of raw data leakage.

[0028] During the aggregation process, the system injects random noise conforming to a Gaussian distribution (μ=0, σ=0.01) into the global gradient. This differential privacy treatment prevents attackers from reverse engineering the training data of individual participants. Even if they obtain the aggregated model parameters, they cannot identify a specific user's biometrics or transaction history.

[0029] The dynamic feature library is intelligently updated, and the system assigns a dynamic score to each attack feature based on detection confidence (model prediction probability), feature freshness (recent frequency), and cross-modal consistency (such as voice and lip matching). When the feature library reaches a preset capacity (for example, 100,000 features), a "survival of the fittest" mechanism automatically activates: the system replaces the lowest-scoring, outdated data in the library with currently detected high-scoring features (such as the frequency domain features of new AI-powered voiceprint forgeries), ensuring that the feature library always covers the latest attack methods.

[0030] Multi-level authentication linkage response: when the transaction risk score exceeds the threshold (such as voiceprint matching degree <0.7 and IP address abnormality), the system immediately triggers the defense interlock mechanism: first, real-time interception is performed through SMS verification code (response time <200ms). If the user passes the verification, it will be transferred to manual review (automatically retrieving previous transaction videos stored in blockchain for comparison). At the same time, the current attack features are synchronized to the global feature library, forming a closed-loop defense system of detection-interception-learning.

[0031] Federated learning (S6) and dynamic feature library management (S7) are deeply coupled through a closed data loop. The encrypted, aggregated global model in federated learning (such as the ResNet-50+Transformer dual-stream network) continuously improves attack detection accuracy. Its output, including forgery probability and cross-modal discrepancy analysis, directly serves as the scoring basis for the dynamic feature library. For example, if the model detects frequency-domain noise characteristics (such as abnormal 48kHz harmonics) in a new deepfake video, it immediately assigns a high score to that feature and stores it in the feature library. New attack patterns with high scores in the dynamic feature library (such as AI-generated pupil reflex distortion) are added to the local training set as adversarial examples by federated learning participants. This real-time data feed mechanism enables the model to automatically strengthen its ability to recognize the latest attack methods in the next round of training, forming a reinforcement learning cycle of detection, feature extraction, model enhancement, and re-detection.

[0032] The Paillier-encrypted gradients (such as the ResNet-50 convolution kernel weight adjustment) work together with the injected Gaussian noise (such as random perturbations of μ=0, σ=0.01) during the aggregation stage, ensuring that the facial, voiceprint, and other biometric data of each institution's customers are not leaked, while also preventing attack features in the dynamic feature library (such as lip-speech synchronization error data) from being associated with specific users, thereby achieving privacy compliance throughout the entire process from model training to feature application.

[0033] This embodiment continuously optimizes model performance through a collaborative federated learning framework. After each participant trains the detection model locally, encryption technology is used to securely aggregate model parameters. Random noise is added during the update process to effectively prevent data privacy leaks. The system automatically calculates a comprehensive score based on the confidence level of the current detection result. When the feature library capacity reaches its upper limit, historical feature data with lower scores is preferentially replaced to maintain the timeliness of attack pattern identification. High-risk transactions are automatically triggered for secondary authentication processes such as SMS verification codes or manual review, forming a dynamically optimized security protection mechanism.

[0034] In a preferred embodiment, the specific process of dynamic timing alignment in S2 includes: S21: Calibrate the clock of the multimodal data acquisition device through the PTP protocol to control the synchronization error within ±1ms; S22: Using a buffer to store data of each modality, and performing optical flow interpolation compensation for missing frames after a timeout; S23: Calculate the lip-speech synchronization error Δt. If Δt>50ms, it is determined to be a forgery attack.

[0035] Clock calibration (S21): The Precision Time Protocol (PTP) is used to unify the internal clocks of all cameras, microphones, and other acquisition devices to ensure that the time deviation between video frames and audio sampling points does not exceed 1 millisecond. Data buffering and compensation (S22): The system sets up a temporary storage area (buffer) to receive data from each device. If a device has a transmission delay (such as a camera frame drop), the content of the missing frame is intelligently inferred based on the optical flow motion trajectory of adjacent frames (such as the direction and speed of lip movement) to avoid incomplete data affecting detection. Lip reading and speech synchronization detection (S23): Real-time calculation of the matching delay (Δt) between the lip opening and closing movement and the speech spectrum. If the delay exceeds 50 milliseconds, it is judged as a forgery (when a real person pronounces, the lip shape and speech are naturally synchronized, which is difficult for deep fake technology to accurately simulate).

[0036] In a preferred embodiment, reference Figure 3 , the training process of the two-stream GAN detector in S4 includes: S41: The generator generates fake videos and synchronized speech based on StyleGAN3, constraining the Fréchet Inception Distance between the generated samples and the real data to be ≤15; S42: The detector uses a focal loss function to jointly optimize the classification loss and reconstruction loss. The formula is: Where λ=0.1, G is the autoencoder; the joint loss function of the detector, its core is to improve the forgery detection ability through dual supervision; the reconstruction task forces the detector to understand the essential structure of the image rather than memorize the forgery cases in the training set; λ=0.1 means that classification is the main method and reconstruction is the auxiliary method to avoid the model falling into local optimality.

[0037] S43: Alternately train the generator and detector until the detector's recognition accuracy for the latest generated samples is ≥ 95% and remains stable for 3 rounds.

[0038] Generator adversarial training: The generator (StyleGAN3) continuously generates realistic fake videos and accompanying audio, but the system requires that the fidelity difference (FID score) between the generated samples and the real data does not exceed 15 points (the lower the FID, the more realistic it is). Detector dual-objective optimization: The detector performs two tasks simultaneously: ① Classification task (determine whether the input is real or fake, and the loss value is 0). Reflecting classification error); ② Reconstruction task (autoencoder G reconstructs input image I into G(I), and the loss value reflects the reconstruction error). A weighted combination of the two (λ = 0.1) forces the detector to deeply detect forgeries. Adversarial iterative training: The generator and detector are alternately upgraded, improving the generator's forgery capabilities and the detector's recognition capabilities. Training terminates (reaching dynamic equilibrium) when the detector achieves a recognition accuracy of ≥ 95% for the newly generated samples over three consecutive rounds of training.

[0039] The autoencoder G inputs: image I (may be real or fake); output: reconstructed image G(I). Input: Detector predicted labels and true labels; Output: Classification error. λ controls the weight of the reconstruction loss in the total loss, balancing the importance of the two tasks. Temporal alignment ensures temporal consistency of multimodal data (for example, lip movements strictly match speech), providing a foundation for accurate cross-modal analysis in the two-stream GAN detector. The GAN detector, through adversarial training, generates realistic forged samples, which feed back into the temporal alignment module to optimize the error threshold (for example, gradually tightening Δt from 50ms), forming a synergistically enhanced defense system.

[0040] In a preferred embodiment, the process of generating structured rules in S5 includes: S51: Using Legal BERT to parse regulatory documents and extract IF-THEN logic rules; S52: Mapping the transaction amount, IP risk score, and voiceprint matching degree into a risk level; S53: If the risk score exceeds the threshold, the transaction is automatically intercepted and a blockchain audit log is generated.

[0041] The compliance verification layer features an intelligent rules engine, enabling automated oversight through a three-tiered mechanism. First, intelligent regulatory analysis utilizes a legal BERT model trained specifically for legal texts. This model deeply analyzes regulatory documents such as the People's Bank of China's Anti-Money Laundering Regulations and the Cross-Border Payment Guidelines, automatically extracting key constraints and converting them into executable logical rules (for example, "IF a single transaction amount exceeds 500,000 RMB THEN two-factor authentication is required"). Multi-dimensional risk quantification integrates transaction characteristics in real time: amount anomaly, measured by comparing it to the user's historical transaction average (e.g., a sudden increase of more than three times the amount); IP risk score, determined based on a geolocation database (e.g., IPs from high-risk overseas regions receive a score of 0.8); and voiceprint matching, calculated using the ECAPA-TDNN voiceprint model (e.g., a score of 0.75 falls below the threshold). If the combined score exceeds the threshold (e.g., 0.9), a resolution mechanism is triggered. For on-chain resolution auditing, when a high-risk transaction is intercepted, the system automatically generates a structured log (JSON format) containing a timestamp, risk indicators, and resolution rationale. This log is encrypted with SHA-256 and written to a blockchain (e.g., Hyperledger Fabric), creating an immutable chain of regulatory audit evidence. The rules analyzed by legal BERT (such as large transaction restrictions) directly define the risk scoring threshold of S52, while the logs stored in the blockchain reversely verify the effectiveness of the rule execution, forming a complete regulatory closed loop of rule formulation, dynamic scoring, and disposal of evidence.

[0042] In a preferred embodiment, the parameter update process of federated learning in S6 includes: S61: Each participant trains the detection model locally, performs Paillier encryption on the gradient parameters, and then uploads them; S62: The aggregation server decrypts the gradient and updates the global model. The formula is: Input encrypted gradient set { , , ..., }、Current global parameters (model weights before update) and learning rate η (hyperparameter, preset); output updated global parameters (adjusted model weights). Where, It is the updated global model parameters (such as the neural network weight matrix) that integrates the knowledge of all participants to form a more powerful deep fake detection model. Represents the learning rate, ranging from 0.001 to 0.1, which controls the parameter update step. The denominator of the gradient average is calculated for the number of institutions participating in federated learning (e.g., 10 banks) to balance the contributions of all parties. The local model gradient of the i-th participant. It is the gradient ciphertext processed by the Paillier encryption algorithm. The Paillier decryption operation converts the aggregated ciphertext gradients back to plaintext. It takes as input the ciphertext sum of the encrypted gradients of each participant and outputs the plaintext sum of all the gradients of the participants.

[0043] Aggregation in encrypted state ( ) Ensure gradients within individual institutions (such as the characteristic parameters of detecting a certain type of forged video) are not leaked, and the average gradient fuses multiple knowledge (such as the forged pupil reflection features discovered by Bank A + the abnormal speech spectrum captured by Bank B), making the global model It can identify more variant attacks. The learning rate η controls the update amplitude to avoid model oscillation caused by gradient mutation. Assume that three banks participate in federated learning: Bank A’s gradient : Enhanced detection of AI-generated ear contour blur; Bank B gradient : Optimizing mechanical noise recognition in speech spectra; Bank C gradient :Improve the sensitivity to abnormal lighting and shadows of virtual background. After calculation by formula, the updated It will have these three types of detection capabilities at the same time, and no party will be able to infer the specific training data of other banks.

[0044] S63: Compressing the global model into MobileViT via knowledge distillation.

[0045] After each bank branch trains the dual-stream detection model using local transaction data, it converts the model's adjusted parameters (such as changes in neural network weights) into a cipher using the Paillier encryption algorithm. This encryption technique allows data to be transmitted in ciphertext, making the original parameters indecipherable even if intercepted, ensuring that customer biometrics (such as voiceprints) are not leaked. Secure aggregation updates: After receiving the encrypted parameters from all branches, the central server applies the unique mathematical properties of the Paillier algorithm to directly perform a weighted average calculation on the ciphertext. After aggregation, unified decryption is performed to obtain a global model parameter update that integrates knowledge from multiple sources. This is similar to fusing the different forgery features identified by each branch (such as AI voice-changing detected by Branch A or pupil reflection anomalies captured by Branch B) into a more powerful detection model. Lightweight model deployment: The updated global model's detection capabilities are transferred to a streamlined version of the MobileViT model using knowledge distillation. This process, similar to distilling expert experience into an operational manual, reduces the model size by 70%. This process maintains original detection accuracy while adapting to the real-time detection requirements of edge devices such as mobile apps, ensuring that users can quickly complete facial liveness detection when transferring money. Technical synergy: Encrypted transmission ensures data privacy, secure aggregation enables knowledge sharing, and model compression solves implementation difficulties. The three form a complete technical chain from data security, algorithm optimization to application deployment.

[0046] In a preferred embodiment, the method for calculating the comprehensive score in S7 includes: S71: Calculate the total confidence score based on the predicted entropy of the current test result and the historical average entropy; S72: taking the ratio of the total confidence score to the preset threshold as the comprehensive score; S73: If the score is ≥ the lowest score in the dynamic feature library, replace the corresponding feature and update the model.

[0047] The dynamic scoring mechanism enables intelligent evolution of the feature library and risk management through a three-tiered decision-making process. First, for abnormal fluctuation detection, the system calculates the predicted entropy value of the current transaction in real time (i.e., the confidence level of the model's judgment, with lower values indicating greater certainty). By comparing the deviation of the current entropy value from the historical 7-day average entropy value (e.g., a sudden 30% drop in the current entropy value), it identifies abnormally high-confidence detection results (possibly indicating a new type of forgery attack). For risk quantification, the total confidence score (e.g., the abnormal entropy ratio + the IP risk score) is proportionally converted to a preset threshold (e.g., 1.2 times the baseline value) to generate a comprehensive score from 0 to 10. For example, when a new deepfake video is detected, the system calculates a score of 8.5 (threshold = 7) due to the high model certainty (entropy value 0.3 < historical average 0.5) and the unusual IP location. The feature library is dynamically updated. When the feature library is full (e.g., reaching 100,000 entries), the system automatically initiates a feature selection process: a high-scoring feature (e.g., a new AI voice-changing spectrum feature with a score of 8.5) is replaced with the lowest-scoring feature in the library (e.g., a traditional Photoshop feature with a score of 4.2). This process also triggers a secondary verification process (e.g., requiring users to complete a blink and head turn liveness test), and the new features are synchronized with the federated learning model for enhanced training. An entropy comparison mechanism can identify model blind spots (e.g., a sudden surge of high-confidence false positives, indicating a new attack). A score replacement strategy ensures that the feature library continuously captures the latest attack patterns (similar to antivirus software virus database updates). Model updates and secondary verification provide a dual defense. Even if the detection model temporarily misses a detection, manual review can still intercept risky transactions.

[0048] In a preferred embodiment, the constraints of the generator include: limiting the Wasserstein distance between the generated sample and the real data to ≤ 0.1, as follows: .

[0049] The Wasserstein distance formula expresses the Wasserstein distance (bulldozer distance) between the true data distribution P_r and the generated data distribution P_g, quantifying the degree of difference between the two distributions (the smaller the value, the closer the generated data is to the truth).

[0050] is the lower bound, is the set of all possible joint distributions, whose marginal distributions are and , assuming It is a collection of real face pictures. is a set of fake faces created by the generator, Each γ in represents a "pairing method of fake images and real images". A specific joint distribution (i.e., a pairing rule for fake images and real images) defines how to match generated data with real data (such as pairing a fake face with the most similar real face). Under the joint distribution γ, take the expected value of the sample pair (x, y). Input all pairs of fake images and real images (x, y), where x comes from , y comes from , output the average transportation cost. is the Euclidean distance between samples x and y. For example, x is a real face image (pixel matrix) and y is a generated fake image. Calculate the color difference between the two. The entire formula shows that this distance measure will generate data distribution "Transfer" to the real distribution The minimum average cost is determined by finding the most labor-saving transportation solution among all possible real and fake data pairings. For each pair of real and fake data, the difference metric (e.g., the pixel difference between image A and image B) is calculated, and the average transportation cost of all pairs is taken as the minimum.

[0051] Generator training: By minimizing the value of W, the generator (such as StyleGAN3) is forced to produce more realistic fake data. Constraining W to ≤ 0.1 ensures the quality of adversarial examples and enables detectors (such as ResNet-50) to learn subtle forgeries. Adversarial training of generation and detection improves overall robustness. For example, when the detector identifies new forged features, the generator adjusts its strategy to reduce the value of W, triggering a new round of attack and defense escalation.

[0052] The generator (StyleGAN3) undergoes rigorous quality control when creating fake data: the system calculates the distribution difference (Wasserstein distance) between the generated samples and the real face database in real time, enforcing a limit of no more than 0.1 (0 represents perfect match, 1 represents significant deviation). This effectively imposes a "limiter" on the generator, ensuring that the fake videos it generates are realistic enough for model training (for example, by simulating pupil reflections) but not excessively distorted, causing the detector to mislearn incorrect features.

[0053] In a preferred embodiment, the optimization process of edge detection includes: S81: Use knowledge distillation to compress ViT-Base into MobileViT, and the loss function is: Where α=0.7, T=3; S82: Deploy the quantized INT8 model to the NPU, with a single detection latency of ≤60ms.

[0054] is the cross entropy loss term, is the KL divergence term, is the total loss value of knowledge distillation, which is used to simultaneously optimize the student model’s prediction of the true label and the teacher model’s knowledge transfer. To balance the weight coefficient (α=0.7), control the weight ratio of cross entropy loss and KL divergence, is the cross entropy loss function, is the original output of the student model (MobileViT), is a real label, such as the risk level in financial transactions (0 = normal, 1 = high risk), is the temperature parameter ( =3), softening the output distribution of the teacher model so that it contains information about the relationship between categories, is the Kullback-Leibler divergence, is the probability distribution of the teacher model (ViT-Base) at temperature T, is the probability distribution of the student model (MobileViT) at temperature T. =0.7: 70% weight is allocated to true label learning (to ensure basic accuracy), 1− =0.3: 30% weight is used for knowledge transfer (absorbing the generalization ability of the teacher model) ， If the teacher model finds the lip tremor feature of a certain type of fake video (not reflected in the label), the student model can indirectly learn this feature through the KL term.

[0055] Using knowledge distillation technology, a large visual Transformer model (ViT-Base) is compressed into a mobile-specific model (MobileViT), allowing the original large model to guide the smaller model in learning key discriminative features (such as facial texture anomalies). The model (and data) are converted to INT8 and deployed on a neural network processing unit (NPU), reducing the time required for single liveness detection to under 60 milliseconds, meeting the requirements of real-time transaction verification. Neural network parameters are compressed from high-precision floating-point numbers (e.g., 32-bit) to 8-bit integers, reducing model size and improving computational speed while maintaining the ability to recognize key features (such as facial contours and voiceprint spectra). The quantized model is then run on a dedicated neural network processing unit (NPU). For example, with NPU chips installed in bank ATMs, deepfake detection (such as AI-powered face-swapped video recognition), which previously required cloud-based processing, can now be performed directly on-device. The INT8 quantized model accurately captures forgery artifacts (such as unnatural eye movements) while preventing sensitive biometric data from being uploaded to the cloud, balancing efficiency and privacy.

[0056] The generator's quality constraints ensure the effectiveness of adversarial examples (for example, generating fake videos that comply with the Wasserstein constraint). The lightweight edge model uses these high-quality examples to train a rapidly deployable detector, forming a closed-loop generation-detection optimization. For example, newly created fake face data from the generator is immediately used for incremental training of the MobileViT model, continuously improving the defense capabilities of edge devices.

[0057] In a preferred embodiment, the rules for cross-modal contradiction analysis include: Rule 1: If the voiceprint matching degree is less than 0.8 and the IP address is overseas, a high-risk alert is triggered; Rule 2: If the transaction amount is greater than 3 times the user's historical average, manual review is mandatory.

[0058] Rule 1 (Voiceprint-IP Correlation Detection) triggers a compound risk alert when the system detects that the similarity between a user's voiceprint and a reserved sample is less than 80% (e.g., a mismatch in voice rhythm or pitch), and the transaction request originates from an overseas IP address (e.g., a US IP accessing a Chinese account). This dual anomaly (biometric mismatch + geographic anomaly) effectively identifies AI voice-changing attacks, such as those where an attacker impersonates a user by forging a voiceprint but exposes risks due to the use of an overseas proxy server.

[0059] Rule 2 (Detecting Sudden Changes in Transaction Behavior) compares the current transaction amount with the user's three-month average transaction amount in real time. If a single transaction suddenly increases by more than three times (for example, a user who transfers 50,000 yuan per month suddenly initiates a 150,000 yuan transaction), the transaction is automatically transferred to a human agent for review. This mechanism addresses unusual fund transfers following account theft. Even if an attacker forges a video to pass facial verification, large, unusual transactions will still be caught.

[0060] Rules work together to prioritize risk: If only Rule 1 or Rule 2 is triggered, the system implements intermediate prevention and control measures (such as SMS verification); if both are triggered simultaneously, the account is immediately frozen and a forensic investigation is initiated. A closed-loop defense system: Manual review results are fed back to the voiceprint database (Rule 1) and user profile (Rule 2). For example, if a large transaction is confirmed to be a normal down payment for a home purchase, the user's transaction threshold is automatically raised to 5 times the average to prevent false interception.

[0061] In a preferred embodiment, data desensitization includes: retaining the eye area of facial images and applying a Gaussian blur (σ=5); partially masking the account number in text commands (e.g., 6214****); and extracting MFCCs from voice features before encrypting them for transmission. The system partially blurs facial photos, retaining only the eye area (used for blinking detection during liveness detection) and applying a Gaussian blur (similar to a mosaic effect) to sensitive areas such as the cheeks and mouth. When users take selfies for verification, the nose and chin are blurred, but the eye contour remains clearly discernible, meeting anti-fraud detection requirements while preventing facial data theft.

[0062] A real-time detection and defense system for financial deep fakes based on a multimodal large model, such as Figure 2 Shown, including: Multimodal input module: used to collect text instructions, facial images, and voice recordings, and perform desensitization processing at the edge node; Timing alignment module: This module uses the dynamic time warping (DTW) algorithm to align multimodal data (using the dynamic time warping (DTW) algorithm to align the time axis of multimodal data to achieve synchronous matching of voice and video signals) and calculates the lip-sync error. Dynamic risk modeling module: contains a real-time updated risk feature library that stores attack pattern features and frequency domain noise fingerprints; Deepfake Detection Module: This includes a dual-stream GAN detector and a cross-modal contradiction analysis engine, outputting forgery probability and risk score; Compliance Verification Module: Analyzes financial regulatory rules and generates structured interception strategies, connecting to the blockchain audit system; Federated learning module: uses Paillier encryption to aggregate gradients and protects data security through differential privacy; Real-time defense module: Dynamically updates the signature database based on comprehensive scores, triggering SMS authentication or transaction interception.

[0063] When a user initiates a transaction, the multimodal input module simultaneously collects facial video, voice commands, and transaction text (such as the transfer amount). Edge computing nodes then blur the face (retaining the eyes), convert the voice into a signature code, and partially hide the account number (e.g., 6214****), achieving simultaneous data collection and privacy protection. The timing alignment module uses a dynamic regularization algorithm to align the timeline of voice and lip movements with millisecond accuracy. For example, if the voice command "transfer" is detected to be three frames (approximately 100ms) behind the lip movement, the module immediately flags a timing anomaly. The dynamic risk modeling module integrates the latest attack signature libraries (such as the spectral patterns of new AI voice mutators) in real time, synchronizing updates from the security intelligence platform every two hours. If the frequency domain noise in a transaction matches a "deepfake voiceprint" in the signature library with a match greater than 90%, the risk level is automatically increased. The deepfake detection module uses two collaborative streams: In the image stream, ResNet-50 analyzes facial artifacts (such as unnatural pupil reflections) in a single face frame; in the video stream, Transformer captures motion anomalies between consecutive frames (such as blinking frequencies that don't conform to biological patterns); and in fusion decision-making, a cross-modal attention mechanism assigns higher weight to high-risk features (e.g., lip syncing + pupil reflection anomalies = 20% increase in overall risk score). The compliance verification module automatically parses documents such as the central bank's anti-money laundering regulations and generates enforceable rules (e.g., requiring secondary authentication for cross-border transfers exceeding 500,000 yuan per day). When high-risk transactions are detected, a blockchain interface is immediately invoked to log the operation (including timestamps, risk indicators, and action rationale). In the federated learning module, each bank branch trains a model using local data and then uploads encrypted parameters to a central server. The server aggregates and generates a global model, which is distributed to all nodes, enabling knowledge sharing while ensuring customer privacy (e.g., customer facial data from Bank A cannot be leaked to Bank B). The real-time defense module automatically updates defense strategies based on dynamic scoring. When the feature inventory is full, outdated features will be replaced first (such as eliminating old features based on PS photo editing and adding AI face-changing detection parameters), and multi-level authentication will be triggered for high-risk transactions (such as SMS verification + manual video review).

[0064] In a preferred embodiment, the dual-stream GAN detector includes: image stream: extracting spatial artifact features based on ResNet-50; video stream: encoding inter-frame temporal features based on Transformer; fusion layer: weighting multimodal features through a cross-modal attention mechanism.

[0065] The two-stream GAN detector builds a three-dimensional defense against deepfake detection through dual-dimensional verification, combining static feature capture and dynamic behavior analysis. The image stream (ResNet-50) focuses on examining the details of a single frame, much like a forgery examiner examining an ID photo with a magnifying glass. Detection focuses on facial skin texture, pupil reflection consistency, and edge blur. It identifies common artifacts in generative AI, such as unnatural ear contours and jagged hairlines. It can detect unusual light spots at the bridge of the nose in forged faces (this area should transition smoothly in real faces). The video stream (Transformer) analyzes the plausibility of movement between consecutive frames, similar to a referee watching a slow-motion replay. Detection focuses on blink frequency, head rotation trajectory, and duration of micro-expressions. It detects temporal paradoxes in deepfake videos (for example, the fixed 2-second interval between AI-generated blinks contradicts human randomness). It can also detect changes in ear shadows when a person turns their head in a forged video, which contradict physical lighting laws. Cross-modal fusion (the intelligent decision-making hub) dynamically integrates dual-stream evidence through an attention mechanism, achieving a detection efficiency that is superior to 1+1. Weighting logic: If a highly suspicious artifact is detected in the image stream (80% confidence) and the video stream is normal (30% confidence), the final risk weighting favors the image stream. Defense upgrade: When new attacks simultaneously deceive single-modal detection (such as a high-precision forged single frame combined with legitimate motion), the fusion layer exploits cross-modal discrepancies (such as a phase shift between lip movements and the speech spectrum) to defeat defenses.

[0066] In a preferred embodiment, the specific configuration of the federated learning module includes: For local training by participants, the ECAPA-TDNN voiceprint model is adopted; the aggregation period is 2 hours, and CUDA is used to accelerate encrypted computing; the global model is compressed into MobileViT through knowledge distillation. The federated learning module constructs a secure and collaborative defense knowledge network through distributed training and efficient compression technology. Local voiceprint special training (ECAPA-TDNN model), each bank branch uses local customer voice data to train a professional voiceprint recognition model (ECAPA-TDNN). This model is like a "voiceprint fingerprint expert", focusing on learning personalized features in the voice (such as pronunciation rhythm, formant distribution). For example, it can accurately identify the unique habit of a certain customer reading "two" as "èr" instead of the dialect "liǎng". Encrypted and efficient aggregation (CUDA acceleration), the global model upgrade is automatically started every 2 hours. Encrypted upload: Each branch encrypts the trained parameters (such as voiceprint feature extraction rules) into the "password book" format. Lightning aggregation: Using the GPU graphics card (CUDA) for parallel decryption computing, the data fusion of dozens of institutions can be completed within 10 minutes. Immediate effect: The updated model can identify new types of attacks (such as the high-frequency noise characteristics of AI synthesized voices discovered by Bank A). Lightweight and intelligent transplantation (MobileViT compression), through knowledge distillation technology, "slim down" the complex global model into a lightweight version (MobileViT) that can be carried by mobile phones: Volume reduction and accuracy retention: Through the "old brings new" training mechanism, the voiceprint recognition accuracy rate remains above 95%. The mobile APP can complete the voiceprint comparison within 0.5 seconds when calling this model (such as the voice transfer verification of WeChat Pay).

[0067] In a preferred embodiment, the compliance verification module includes: a legal BERT parsing engine that converts regulatory clauses into JSON format rules; a rule executor that calls the blockchain interface to record logs according to the risk score; a dynamic policy library that synchronizes and updates from the regulatory agency API every 6 hours. The legal BERT parsing engine deeply analyzes the regulatory documents of institutions such as the central bank and the banking and insurance regulatory commission, and converts the articles into executable codes. When the transaction risk score triggers the rule threshold (such as voiceprint matching degree < 0.7 + amount exceeding limit), the rule executor automatically performs a three-link operation. Real-time interception: Freeze suspicious transactions and send SMS verification. Blockchain evidence storage: Generate encrypted logs (such as SHA-256 hash values) for risk indicators, disposal bases, etc., and write them into the distributed ledger. Regulatory reporting: Send early warning filings to the monetary authority system through a dedicated API. The dynamic policy library automatically connects to the regulatory agency data port every 6 hours to capture the latest policy changes. Incremental update: Only download the changed clauses (reducing the data transmission volume by 90%). Compatibility detection: Verify the conflicts between the new rules and the existing business systems.

[0068] In a preferred embodiment, the real-time defense module includes: an edge NPU accelerator that deploys a quantized detection model; a buffer management unit that handles multimodal data delays and frame loss; and a defense strategy selector that dynamically adjusts authentication levels based on confidence scores. The real-time defense module leverages a three-layered approach: edge computing, data repair, and intelligent response, to achieve millisecond-level risk management capabilities. The edge NPU accelerator, built into dedicated chips in devices like ATMs and mobile phones, carries a lightweight detection model (such as the 25MB MobileViT). This acts as a "sharp eye" for the device, enabling rapid facial liveness detection (e.g., determining whether a user is real or a photo). The buffer management unit automatically activates an optical flow compensation algorithm when network fluctuations cause temporal misalignment between the video and voice streams (e.g., voice precedes lip movements by 200ms). This algorithm uses the motion trajectories of adjacent frames to intelligently infer missing frames (e.g., predicting the intermediate states of a user's lips from closed to open), ensuring the integrity of multimodal data. A defense strategy selector dynamically adjusts the protection level based on real-time risk scores: Low risk (score < 6): SMS verification code authentication; Medium risk (6 ≤ score < 8): Voice and facial recognition dual authentication; High risk (score ≥ 8): Mandatory video manual review and temporary account freeze. For example, if abnormal pupil reflections are detected in a fake video (score 7.5), dual authentication is immediately initiated. If an overseas IP address is also detected (score 9.2), the account is directly transferred to manual verification and frozen.

[0069] The present invention builds a secure input barrier through multimodal data collection and edge desensitization, and uses dynamic timing alignment technology to ensure the spatiotemporal consistency of cross-modal data, providing a precise analysis basis for the dual-stream GAN detector; the dynamic risk modeling layer injects the latest threat intelligence through the real-time updated attack feature library, driving the continuous evolution of the detection model; the deep fake detection module integrates image artifact analysis and video timing verification, and combines cross-modal contradiction identification to form a three-dimensional defense; the compliance verification layer converts legal clauses into executable policies and realizes audit traceability through blockchain; the federated learning framework realizes multi-party knowledge sharing in an encrypted environment, and cooperates with lightweight edge deployment to form end-cloud collaboration; the real-time defense module implements hierarchical response according to dynamic scoring, and synchronously updates the feature library and defense strategy. Each module is tightly coupled through a closed-loop data flow and feedback mechanism to achieve full-link intelligent protection from attack identification, model optimization to compliance disposal, effectively responding to the evolving deep fake threats in financial scenarios.

[0070] Although the preferred embodiments of the present invention have been described in detail, those skilled in the art may make other changes and improvements to these embodiments after understanding the basic innovative concepts. Therefore, the appended claims are intended to cover the preferred embodiments and all changes and improvements that fall within the scope of the present invention. The foregoing is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A real-time detection and defense method for financial deep fakes based on a multimodal large model, characterized by: Includes the following: S1: Acquire multimodal data in financial transaction scenarios and perform edge desensitization on each modal data; S2: Dynamically align the multimodal data using a dynamic time warping algorithm to align the speech and video timelines and calculate the synchronization error between lip movements and speech. S3: Input the time-aligned features into the dynamic risk modeling layer and generate dynamic risk features based on the real-time updated risk feature library. The risk feature library pulls the latest attack pattern features from the threat intelligence platform every X hours. S4: Analyze dynamic risk features through the deepfake detection layer's two-stream GAN detector, which includes a ResNet-50 for the image stream and a Transformer for the video stream, outputs forgery probabilities, and identifies cross-modal inconsistencies. S5: Input the detection results into the compliance verification layer, parse the regulatory documents through legal BERT and generate structured rules, perform real-time transaction interception and blockchain logging.

2. A method for real-time detection and defense of financial deep fakes based on a multimodal large model according to claim 1, characterized in that: Also includes: S6: Update the detection model parameters based on the federated learning framework, use Paillier homomorphic encryption to aggregate gradients, and inject Gaussian noise to ensure differential privacy. S7: Calculate the comprehensive score of the detection results. If the capacity of the dynamic feature library exceeds the threshold, replace the historical features based on the score and trigger secondary authentication.

3. The method for real-time detection and defense of financial deep fakes based on a multimodal large model as claimed in claim 1, characterized in that: The specific process of dynamic timing alignment in S2 includes: S21: Calibrate the clock of the multimodal data acquisition device through the PTP protocol to control the synchronization error within ±1ms; S22: Using a buffer to store data of each modality, and performing optical flow interpolation compensation for missing frames after a timeout; S23: Calculate the lip-speech synchronization error Δt. If Δt>50ms, it is determined to be a forgery attack.

4. The method for real-time detection and defense of financial deep fakes based on a multimodal large model according to claim 1 is characterized in that: The training process of the two-stream GAN detector in S4 includes: S41: The generator generates fake videos and synchronized speech based on StyleGAN3, constraining the Fréchet Inception Distance between the generated samples and the real data to be ≤15; S42: The detector uses a focal loss function to jointly optimize the classification loss and reconstruction loss; S43: Alternately train the generator and detector until the detector's recognition accuracy for the latest generated samples is ≥ 95% and remains stable for 3 rounds.

5. A method for real-time detection and defense of financial deep fakes based on a multimodal large model according to any one of claims 1 to 4, characterized in that: The process of generating structured rules in S5 includes: S51: Using Legal BERT to parse regulatory documents and extract IF-THEN logic rules; S52: Mapping the transaction amount, IP risk score, and voiceprint matching degree into a risk level; S53: If the risk score exceeds the threshold, the transaction is automatically intercepted and a blockchain audit log is generated.

6. The method for real-time detection and defense of financial deep fakes based on a multimodal large model according to claim 2 is characterized in that: The parameter update process of federated learning in S6 includes: S61: Each participant trains the detection model locally, performs Paillier encryption on the gradient parameters, and then uploads them; S62: The aggregation server decrypts the gradient and updates the global model; S63: Compressing the global model into MobileViT via knowledge distillation.

7. The method for real-time detection and defense of financial deep fakes based on a multimodal large model according to claim 2 is characterized in that: The calculation method of the comprehensive score in S7 includes: S71: Calculate the total confidence score based on the predicted entropy of the current test result and the historical average entropy; S72: taking the ratio of the total confidence score to the preset threshold as the comprehensive score; S73: If the score is ≥ the lowest score in the dynamic feature library, replace the corresponding feature and update the model.

8. The method for real-time detection and defense of financial deep fakes based on a multimodal large model as claimed in claim 1, characterized in that: The rules for cross-modal contradiction analysis include: Rule 1: If the voiceprint matching degree is less than 0.8 and the IP address is overseas, a high-risk alert is triggered; Rule 2: If the transaction amount is greater than 3 times the user's historical average, manual review is mandatory.

9. A real-time financial deepfake detection and defense system based on a multimodal large model, characterized by: include: Multimodal input module: used to collect text instructions, facial images, and voice recordings, and perform desensitization processing at the edge node; Timing alignment module: aligns multimodal data using a dynamic time warping algorithm and calculates lip-speech synchronization errors; Dynamic risk modeling module: contains a real-time updated risk feature library that stores attack pattern features and frequency domain noise fingerprints; Deepfake Detection Module: This includes a dual-stream GAN detector and a cross-modal contradiction analysis engine, outputting forgery probability and risk score; Compliance Verification Module: Analyzes financial regulatory rules and generates structured interception strategies, connecting to the blockchain audit system; Federated learning module: uses Paillier encryption to aggregate gradients and protects data security through differential privacy; Real-time defense module: Dynamically updates the signature database based on comprehensive scores, triggering SMS authentication or transaction interception.

10. A financial deep fake real-time detection and defense system based on a multimodal large model according to claim 9, characterized in that: The two-stream GAN detector includes: Image stream: Extract spatial artifact features based on ResNet-50; Video stream: Transformer-based encoding of inter-frame temporal features; Fusion layer: weights multimodal features through a cross-modal attention mechanism.

Images