AI-driven financial big data real-time anomaly detection method and system

Abstract

The present application relates to the technical field of financial big data, and more particularly to an AI-driven financial big data real-time anomaly detection method and system. Each participant iteratively trains local financial transaction data sets to generate local parameter gradient data, which is encrypted using a homomorphic encryption public key and uploaded to an aggregation server along with data distribution characteristics. The aggregation server determines a subset of participants based on the data distribution characteristics and synthesizes global encrypted gradient data in the ciphertext domain and distributes it. Each participant decrypts and updates the local anomaly detection parameter set, and based on the updated parameters, the financial transaction data to be detected is determined to be abnormal and alarm information is generated. This method protects data privacy while achieving efficient real-time anomaly detection.

Description

Technical Field

[0001] This invention relates to the field of financial big data technology, and in particular to an AI-driven real-time anomaly detection method and system for financial big data. Background Technology

[0002] In the field of real-time anomaly detection in financial big data, current practices primarily rely on centralized machine learning models. Financial institutions transmit their local financial transaction data to a centralized data processing platform, which then trains anomaly detection models based on a global dataset. These models typically employ algorithms such as support vector machines, random forests, or deep neural networks to extract transaction features and establish a baseline for normal behavior. Transactions deviating from this baseline are then identified as anomalies. After training, the models are deployed on local or cloud servers of each participant for real-time transaction monitoring.

[0003] Centralized approaches require all participants to aggregate sensitive financial data at a single center, making data transmission vulnerable to theft. If the central server is compromised, massive amounts of customer transaction records face the risk of large-scale leakage. Furthermore, the data distribution across different financial institutions may vary significantly; for example, credit card transactions and mortgage transactions have different feature spaces. Centralized models, by forcibly fitting heterogeneous data, result in low detection rates for specific types of anomalies.

[0004] While federated learning frameworks alleviate the problem of sharing raw data, they neglect the impact of the data distribution characteristics of each participant on the quality of model aggregation. All participants are treated equally, without considering the degree of matching between their local and global data distributions. When a participant's data distribution deviates significantly from the mainstream, its uploaded gradients may introduce negative noise, leading to slow convergence or even divergence of the global model. Furthermore, malicious participants may upload forged gradient parameters, compromising model integrity. Existing frameworks lack effective participant screening mechanisms based on data distribution characteristics, making it difficult to guarantee the robustness of anomaly detection. Summary of the Invention

[0005] This invention provides an AI-driven real-time anomaly detection method and system for financial big data, which can solve the problems in the prior art.

[0006] A first aspect of this invention provides an AI-driven real-time anomaly detection method for financial big data, comprising: Each participant constructs a local anomaly detection parameter set based on the local financial transaction dataset, and performs iterative training on the local anomaly detection parameter set using the local financial transaction dataset to generate local parameter gradient data. Each participant uses a homomorphic encryption public key to perform homomorphic encryption operations on the local parameter gradient data to generate encrypted gradient data. Based on the local financial transaction dataset, data distribution features are extracted, and the encrypted gradient data and the data distribution features are uploaded to the aggregation server. The aggregation server determines a subset of participants in the aggregation based on the degree of matching between the data distribution characteristics of each participant. Within the ciphertext domain, it performs homomorphic addition on the encrypted gradient data of each participant in the subset to synthesize global encrypted gradient data, and then distributes the global encrypted gradient data to each participant. Each participant uses a homomorphic encryption private key to perform homomorphic decryption on the received global encryption gradient data to obtain decryption gradient data, and updates the local anomaly detection parameter set based on the decryption gradient data; Each participant performs anomaly detection calculations on the financial transaction data to be detected based on the updated local anomaly detection parameter set to generate anomaly detection results. When the anomaly detection results indicate an anomaly, an alarm message is generated and pushed to the corresponding participant.

[0007] Generating the encrypted gradient data and extracting the data distribution features includes: Gradient encoding operation is performed on the gradient components corresponding to each trainable parameter in the local parameter gradient data, and each gradient component is mapped to the corresponding coefficient position in the plaintext polynomial coefficient space of the fully homomorphic encryption scheme to obtain encoded gradient data. The homomorphic encryption public key is used to perform a fully homomorphic encryption operation on the encoded gradient data to generate encrypted gradient data. The fully homomorphic encryption operation supports homomorphic addition and homomorphic multiplication operations within the ciphertext domain. Multidimensional statistical feature extraction operations are performed on the numerical distribution of each transaction feature dimension related to the anomaly detection task in the local financial transaction dataset to generate the data distribution features. The data distribution features include the distribution mean, distribution variance, distribution skewness of each transaction feature dimension, and the dispersion index of each transaction feature dimension in a preset quantile interval. The encrypted gradient data and the data distribution features are respectively encapsulated in a data structure format. The encrypted gradient data is generated by adding participant identifiers and iteration round identifiers to the encrypted gradient data, and the data distribution features are generated by adding participant identifiers and iteration round identifiers to the data distribution features. The encrypted gradient data and the distribution feature data are then uploaded to the aggregation server via independent secure transmission channels.

[0008] Performing the multidimensional statistical feature extraction operation includes: Nonparametric probability density estimation is performed on the numerical values ​​of each transaction feature dimension. A preset number of equal-width quantile intervals are divided within the value range of each transaction feature dimension. The frequency of samples falling into each interval is counted. The sample frequency of each interval is divided by the total number of samples in that dimension to obtain the sample frequency of each interval, and an empirical distribution histogram of each transaction feature dimension is generated. Based on the empirical distribution histogram, a moment estimation operation is performed to calculate the first-order raw moment of each transaction feature dimension as the distribution mean, the second-order central moment of each transaction feature dimension as the distribution variance, and the third-order central moment of each transaction feature dimension, after standardization, as the distribution skewness. The frequency statistics of each transaction feature dimension in each quantile interval are compared with the preset global benchmark frequency distribution. The cumulative result of the absolute value of the frequency difference in each interval is weighted according to the sample frequency of the interval as the dispersion index. The distribution mean, distribution variance, distribution skewness, and dispersion index are structured and organized in the form of multidimensional feature vectors to form the data distribution features. The data distribution features and the encoded gradient data are then synchronously uploaded to the aggregation server.

[0009] The aggregation server determines a subset of participants for aggregation based on the degree of matching between the data distribution characteristics of each participant and performs homomorphic addition synthesis, including: For each participant's data distribution characteristics, the distribution mean, distribution variance, and distribution skewness of each transaction feature dimension are compared pairwise across participants. The difference in distribution mean, distribution variance, and distribution skewness between each pair of participants in each transaction feature dimension are calculated. The absolute values ​​of the above three differences are summed to generate a multidimensional distribution difference measure between participants. Perform cross-participant difference comparison calculation on the distribution divergence index of each participant, and use the absolute value of the difference as the distribution divergence difference measure value among participants. Then, perform a fusion calculation on the multidimensional distribution difference measure value and the distribution divergence difference measure value to generate a comprehensive distribution difference value among participants. Based on the comprehensive distribution difference value among the participants, the participants are clustered according to the connectivity relationship between the comprehensive distribution difference values, and the union of the participants belonging to the same cluster is determined as the participant subset. Within the ciphertext domain, a homomorphic addition synthesis operation is performed on the encrypted gradient data of each participant in the participant subset. The encrypted gradient data of all participants in the participant subset are accumulated element by element by the ciphertext domain homomorphic addition to generate global encrypted gradient data.

[0010] The determination of the subset of participants in the aggregation also includes updating the subset of participants during iteration and distributing the results to each participant, including: In each round of iterative training, the comprehensive distribution difference value among the participants is recalculated based on the data distribution characteristics uploaded by each participant in this round, and the range of participants included in the participant subset is updated based on the recalculated comprehensive distribution difference value among the participants. The global encrypted gradient data and the participant identification information of the updated participant subset are jointly encapsulated into an aggregation result data packet, and the aggregation result data packet is distributed to each participant; After receiving the aggregation result data packet, each participating party parses the participating party identification information carried in the aggregation result data packet, compares its own participating party identification with the participating party identification information, and determines whether it belongs to the participating party subset. For participants whose participant identifiers belong to the subset of participants, homomorphic decryption is performed on the global encrypted gradient data using the homomorphic encryption private key to obtain decrypted gradient data, and the parameter values ​​in the local anomaly detection parameter set are updated based on the decrypted gradient data. For participants whose participant identifier does not belong to the subset of participants, the parameter values ​​in the current local anomaly detection parameter set are kept unchanged, and the anomaly determination operation is performed using the unchanged local anomaly detection parameter set.

[0011] The system performs anomaly detection calculations on the financial transaction data to be inspected and generates alarm information, including: Based on the updated local anomaly detection parameter set, multidimensional feature transformation operations are performed on each transaction feature dimension of the financial transaction data to be detected to generate a dimensionality-reduced feature representation. Feature reconstruction operations are then performed on the dimensionality-reduced feature representation to obtain reconstructed transaction data. The reconstructed transaction data has the same feature structure as the financial transaction data to be detected. The deviation magnitude between the original values ​​of each transaction feature dimension of the financial transaction data to be detected and the reconstructed values ​​of the corresponding dimensions in the reconstructed transaction data is calculated, and the reconstruction deviation components of each dimension are generated. The reconstruction deviation components of each dimension are fused according to the contribution coefficients of each dimension determined in the parameter training stage to obtain the anomaly judgment result. The anomaly determination result is compared with the preset anomaly determination benchmark. When the anomaly determination result exceeds the preset anomaly determination benchmark, the financial transaction data to be detected is marked as abnormal financial transaction data, and the target transaction feature dimension with the largest reconstructed deviation component value is determined as the anomaly attribution dimension. For transaction records marked as abnormal financial transaction data, the abnormal attribution dimension and transaction attribute information of the transaction record are extracted, and the abnormal attribution dimension and transaction attribute information are encapsulated into alarm information and pushed to the corresponding participants.

[0012] A second aspect of this invention provides an AI-driven real-time anomaly detection system for financial big data, comprising: The local training unit is used by each participant to construct a local anomaly detection parameter set based on the local financial transaction dataset, and to perform iterative training on the local anomaly detection parameter set using the local financial transaction dataset to generate local parameter gradient data. An encrypted upload unit is used by each participant to perform homomorphic encryption operation on the local parameter gradient data using a homomorphic encryption public key to generate encrypted gradient data, extract data distribution features based on the local financial transaction dataset, and upload the encrypted gradient data and the data distribution features to the aggregation server. The aggregation and distribution unit is used by the aggregation server to determine a subset of participants in the aggregation based on the matching degree between the data distribution characteristics of each participant, to perform homomorphic addition on the encrypted gradient data of each participant in the subset of participants in the ciphertext domain to synthesize global encrypted gradient data, and to distribute the global encrypted gradient data to each participant. The decryption and update unit is used by each participant to perform homomorphic decryption operation on the received global encrypted gradient data using the homomorphic encryption private key to obtain decryption gradient data, and update the local anomaly detection parameter set based on the decryption gradient data; The anomaly alarm unit is used by each participant to perform anomaly judgment calculations on the financial transaction data to be detected based on the updated local anomaly detection parameter set, and generate anomaly judgment results. When the anomaly judgment result indicates an anomaly, an alarm message is generated and pushed to the corresponding participant.

[0013] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0015] Homomorphic encryption is employed to achieve end-to-end encrypted interaction and aggregation of gradient data, completely avoiding the risk of plaintext exposure of original transaction data and training parameters. Each participant only needs to upload the encrypted gradient data and data distribution characteristics. The aggregation server completes the global model update within the encrypted domain, and the private key is stored locally on each participant's machine. This mechanism ensures that even if the aggregation server or communication link is attacked, the participants' private data or model parameters cannot be recovered, meeting the high confidentiality requirements of the financial industry for sensitive data such as user transaction information and fund flows.

[0016] By introducing a data distribution feature matching mechanism, the aggregation server can dynamically select participants with similar data distribution characteristics to form an aggregation subset, effectively filtering out heterogeneous noise caused by differences in business, geographical distribution, or time span. This design avoids the model bias problem caused by the indiscriminate aggregation of all participants in traditional federated learning, making global gradient synthesis more focused on high-quality data sources with statistical commonalities. Distribution-aligned collaborative training significantly improves the anomaly detection model's generalization ability to local financial risks and maintains stable detection sensitivity in cross-institutional scenarios.

[0017] Participants implement real-time anomaly detection based on globally updated local models. Computational overhead is concentrated in the lightweight forward inference process, and processing latency for high-concurrency transaction streams is controlled within milliseconds. When a combination of features deviating from normal behavior patterns is detected, the system automatically generates an alert push containing an anomaly transaction identifier, risk level, and timestamp, enabling participating institutions to immediately block suspicious operations. This end-to-end collaborative mechanism breaks through data silos that previously relied on limited data from a single institution for anomaly detection, enabling collaborative identification of complex risks such as cross-institutional black market infiltration and money laundering while ensuring data sovereignty. Attached Figure Description

[0018] Figure 1 A flowchart illustrating an AI-driven real-time anomaly detection method for financial big data. Figure 2 A schematic diagram of the process for determining the subset of participants in the aggregation and performing homomorphic additive synthesis. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0021] refer to Figure 1 and Figure 2 The AI-driven real-time anomaly detection method for financial big data according to embodiments of the present invention includes: Each participant constructs a local anomaly detection parameter set based on the local financial transaction dataset, and performs iterative training on the local anomaly detection parameter set using the local financial transaction dataset to generate local parameter gradient data. Each participant uses a homomorphic encryption public key to perform homomorphic encryption operations on the local parameter gradient data to generate encrypted gradient data. Based on the local financial transaction dataset, data distribution features are extracted, and the encrypted gradient data and the data distribution features are uploaded to the aggregation server. The aggregation server determines a subset of participants in the aggregation based on the degree of matching between the data distribution characteristics of each participant. Within the ciphertext domain, it performs homomorphic addition on the encrypted gradient data of each participant in the subset to synthesize global encrypted gradient data, and then distributes the global encrypted gradient data to each participant. Each participant uses a homomorphic encryption private key to perform homomorphic decryption on the received global encryption gradient data to obtain decryption gradient data, and updates the local anomaly detection parameter set based on the decryption gradient data; Each participant performs anomaly detection calculations on the financial transaction data to be detected based on the updated local anomaly detection parameter set to generate anomaly detection results. When the anomaly detection results indicate an anomaly, an alarm message is generated and pushed to the corresponding participant.

[0022] Generating the encrypted gradient data and extracting the data distribution features includes: Gradient encoding operation is performed on the gradient components corresponding to each trainable parameter in the local parameter gradient data, and each gradient component is mapped to the corresponding coefficient position in the plaintext polynomial coefficient space of the fully homomorphic encryption scheme to obtain encoded gradient data. The homomorphic encryption public key is used to perform a fully homomorphic encryption operation on the encoded gradient data to generate encrypted gradient data. The fully homomorphic encryption operation supports homomorphic addition and homomorphic multiplication operations within the ciphertext domain. Multidimensional statistical feature extraction operations are performed on the numerical distribution of each transaction feature dimension related to the anomaly detection task in the local financial transaction dataset to generate the data distribution features. The data distribution features include the distribution mean, distribution variance, distribution skewness of each transaction feature dimension, and the dispersion index of each transaction feature dimension in a preset quantile interval. The encrypted gradient data and the data distribution features are respectively encapsulated in a data structure format. The encrypted gradient data is generated by adding participant identifiers and iteration round identifiers to the encrypted gradient data, and the data distribution features are generated by adding participant identifiers and iteration round identifiers to the data distribution features. The encrypted gradient data and the distribution feature data are then uploaded to the aggregation server via independent secure transmission channels.

[0023] In the process of generating encrypted gradient data and extracting data distribution features, the processing of local parameter gradient data is a crucial step in the entire federated learning secure aggregation process. After one round of iterative training, the local anomaly detection parameter set generates gradient components corresponding to each trainable parameter. These gradient components are typically floating-point real numbers, and directly applying fully homomorphic encryption operations to them requires first mapping them to the plaintext polynomial coefficient space supported by the fully homomorphic encryption scheme.

[0024] The core objective of gradient coding is to establish a one-to-one correspondence between each gradient component and the coefficient positions of the plaintext polynomial. Taking the CKKS (Cheon-Kim-Kim-Song) fully homomorphic encryption scheme as an example, its plaintext space consists of a polynomial of degree... The polynomial ring is composed of a polynomial ring that can accommodate at most [number] polynomials. There are a complex number of slots. Let the total number of trainable parameters in the local anomaly detection parameter set be... Corresponding to generate The i-th gradient component, denoted as the i-th gradient component. The values ​​of each gradient component are ,in The encoding process will Fill in the plaintext polynomial in sequence. Each coefficient slot, when When the number of slots exceeds the capacity of a single plaintext polynomial, the gradient components are batched according to the slot capacity and encoded into multiple plaintext polynomials to form encoded gradient data. This batch encoding method fully utilizes the SIMD (Single Instruction Multiple Data) parallelism of fully homomorphic encryption schemes, enabling the simultaneous aggregation of multiple gradient components during subsequent homomorphic addition operations in the ciphertext domain, significantly reducing the computational overhead of ciphertext operations.

[0025] After gradient encoding, a fully homomorphic encryption operation is performed on the encoded gradient data using a homomorphic encryption public key to generate encrypted gradient data. The employed fully homomorphic encryption scheme supports homomorphic addition and multiplication operations within the ciphertext domain. This means the aggregation server can directly perform ciphertext addition on encrypted gradient data uploaded by multiple participants without holding a private key, completing gradient aggregation without exposing the plaintext gradient information of each participant in any intermediate step. Fully homomorphic encryption introduces a certain degree of noise during execution, which accumulates gradually with the increase in the number of operations. In practical engineering implementation, encryption parameters (including the polynomial modulus, coefficient modulus chain, and scaling factor) need to be set appropriately to ensure that the noise does not exceed the decryptable threshold within the range of homomorphic operations required to complete aggregation, thereby ensuring that the accuracy of the decrypted gradient values ​​meets the model training requirements.

[0026] The extraction of data distribution features focuses on various transaction feature dimensions relevant to the anomaly detection task within the local financial transaction dataset. Financial transaction data typically includes multiple dimensions such as transaction amount, transaction frequency, transaction time interval, merchant category code, and geographic location offset. The numerical distribution of these dimensions directly reflects the statistical characteristics of the participants' local data. For each transaction feature dimension, its distribution mean and variance are calculated to describe the central tendency and dispersion of the data in that dimension. Simultaneously, the distribution skewness is calculated to characterize the symmetry of the numerical distribution of that dimension. The sign and absolute value of the skewness reflect whether there is a long-tail phenomenon in the data distribution, which is crucial for distinguishing fraudulent transactions, as fraudulent transactions often exhibit significant skewness in specific dimensions. Furthermore, it is necessary to calculate the dispersion index of each transaction feature dimension within a preset quantile interval. The preset quantile interval is typically selected as the interquartile range (i.e., the interval between the 25th and 75th percentiles), using the interquartile range as the dispersion index. This index has strong robustness to outliers and can more stably reflect the main distribution pattern of the data without being affected by extreme abnormal transaction records. The aforementioned multidimensional statistical features together constitute a data distribution feature vector, which is used on the aggregation server to measure the degree of matching between the data distributions of each participant, providing a basis for subsequent selection of a subset of participants to participate in the aggregation.

[0027] In the data encapsulation and uploading stage, encrypted gradient data and data distribution features need to be structured and encapsulated separately to ensure that the aggregation server can accurately identify the data source and training epoch. Participant identifiers and iteration epoch identifiers are appended to the encrypted gradient data. The participant identifier uses a globally unique identifier, and the iteration epoch identifier is the global epoch number of the current federated training. Together, they constitute the metadata header of the encrypted gradient data packet, which is then packaged into a single encrypted gradient data packet. The same participant identifier and iteration epoch identifier are appended to the data distribution features to form a distribution feature data packet. The encrypted gradient data packet and the distribution feature data packet are uploaded to the aggregation server via independent secure transmission channels. Both channels are protected by end-to-end encryption protocols at the transport layer. This design of splitting the two types of data packets into independent channels is based on multiple considerations: firstly, the encrypted gradient data packet is usually much larger than the distribution feature data packet, and independent channel transmission can avoid mutual interference from transmission congestion caused by differences in packet size; secondly, the aggregation server can prioritize receiving and processing the distribution feature data packet, completing the participant subset selection decision before all encrypted gradient data packets arrive, thereby achieving pipelined parallelism of data reception and aggregation decision-making, and shortening the overall aggregation latency.

[0028] At the engineering implementation level, structured encapsulation adopts a standardized binary serialization format. Before transmission, the encapsulated data packets require an integrity checksum. Upon receiving the data packets, the aggregation server first verifies the integrity checksum to confirm that the data packets have not been tampered with or damaged during transmission before proceeding with subsequent parsing and processing. The combination of participant identifiers and iteration round identifiers can also be used to prevent replay attacks. The aggregation server maintains a record set of processed data packet identifiers. Data packets with identifier combinations already present in this record set are discarded, thus preventing malicious nodes from interfering with the aggregation results by repeatedly submitting historical round data packets.

[0029] Performing the multidimensional statistical feature extraction operation includes: Nonparametric probability density estimation is performed on the numerical values ​​of each transaction feature dimension. A preset number of equal-width quantile intervals are divided within the value range of each transaction feature dimension. The frequency of samples falling into each interval is counted. The sample frequency of each interval is divided by the total number of samples in that dimension to obtain the sample frequency of each interval, and an empirical distribution histogram of each transaction feature dimension is generated. Based on the empirical distribution histogram, a moment estimation operation is performed to calculate the first-order raw moment of each transaction feature dimension as the distribution mean, the second-order central moment of each transaction feature dimension as the distribution variance, and the third-order central moment of each transaction feature dimension, after standardization, as the distribution skewness. The frequency statistics of each transaction feature dimension in each quantile interval are compared with the preset global benchmark frequency distribution. The cumulative result of the absolute value of the frequency difference in each interval is weighted according to the sample frequency of the interval as the dispersion index. The distribution mean, distribution variance, distribution skewness, and dispersion index are structured and organized in the form of multidimensional feature vectors to form the data distribution features. The data distribution features and the encoded gradient data are then synchronously uploaded to the aggregation server.

[0030] When extracting data distribution features, nonparametric probability density estimation is performed on the numerical values ​​of each transaction feature dimension in the local financial transaction dataset. Nonparametric probability density estimation does not rely on prior assumptions about the data distribution pattern and can objectively reflect the true distribution of actual transaction data across each dimension. Specifically, within the value range of each transaction feature dimension, a preset number of quantile intervals are divided according to the principle of equal width. The number of intervals can be flexibly configured according to the data volume and accuracy requirements, typically between 20 and 100, to achieve a balance between estimation accuracy and computational efficiency. The frequency of samples falling within each interval is counted, denoted as the _th_ __. There are a total of 1 transaction feature dimensions The nth sample, the th The sample frequency within each interval is Then the sample frequency corresponding to this interval is Arranging the sample frequencies of each interval in interval order constitutes the first interval. An empirical distribution histogram for each transaction feature dimension, which approximates the probability density function of the value in that dimension in the form of a discretized frequency distribution.

[0031] Based on the generated empirical distribution histogram, moment estimation is performed to quantify the distribution characteristics of each transaction feature dimension from the perspective of statistical moments. The first-order raw moment, i.e., the distribution mean, reflects the central tendency of the values ​​in that dimension. Let the first-order moment be... The dimensions are divided into The interval, the first The value of the midpoint of each interval is Then the distribution mean The calculation method is as follows The second central moment, also known as the distribution variance, reflects the degree of dispersion of the values ​​in that dimension relative to the mean. The calculation method is as follows The third central moment, after standardization, serves as the distribution skewness. Skewness metrics capture the degree of asymmetry in a data distribution relative to a symmetrical distribution. In financial trading scenarios, abnormal transactions often lead to significant distribution skewness in certain feature dimensions; therefore, skewness is an important descriptive measure of distribution. Distribution skewness in each dimension The calculation method is as follows The denominator is the cube of the standard deviation, which serves as a normalization function, making the skewness value independent of the dimensions of the original data, thus facilitating comparisons across dimensions and among different participants.

[0032] After calculating the moment statistics, a dispersion index is further introduced to measure the degree of deviation between the local data distribution and the preset global benchmark frequency distribution. The preset global benchmark frequency distribution can be obtained from historical normal transaction data statistics or determined through negotiation among the participants during the initialization phase; it represents the reference distribution pattern that each feature dimension should present under normal transaction behavior. For the [missing information]... Let the number of transaction feature dimensions be the global benchmark frequency distribution at the th dimension. The frequency values ​​in each interval are The frequency of local experience distributed in the same interval is . The absolute value of the frequency difference between the two in this interval is Dispersion index The calculation is performed using a weighted cumulative method with the sample frequency of each local interval as the weight. Specifically: This weighting method makes the more concentrated intervals of the sample contribute more to the dispersion index, thus highlighting the deviation between the main distribution area and the benchmark distribution, rather than being disturbed by abnormal fluctuations in the low-frequency tail intervals. When the distribution pattern of local transaction data highly matches the global benchmark distribution, Approaching zero; when there are a large number of abnormal transactions or data offsets in the local data, This will significantly increase the data quality of participating parties, providing a quantitative basis for subsequent aggregation servers to judge the data quality of participating parties.

[0033] In financial transaction data, common transaction feature dimensions include transaction amount, transaction frequency, transaction time interval, and recipient category distribution. The nonparametric probability density estimation and moment estimation operations described above are performed independently on each dimension to obtain the corresponding distribution mean. Distribution variance skewness of distribution and dispersion index The four types of statistics from all dimensions are concatenated in a fixed order to form a multidimensional feature vector, i.e., the data distribution feature vector. Let the total number of transaction feature dimensions be... Then the dimension of the data distribution feature vector is Its structured organization ensures that the data uploaded by different participants has a unified format, which facilitates the aggregation server to perform cross-participant matching degree calculation after receiving the data.

[0034] The structured data distribution feature vectors and the homomorphically encrypted encoded gradient data are synchronously uploaded to the aggregation server. The data distribution feature vectors are uploaded in plaintext, containing only statistical distribution descriptions and not specific records of the original transaction data, thus preventing privacy leaks of the original data. Upon receiving the data distribution feature vectors from each participant, the aggregation server calculates the similarity or distance between the feature vectors to determine the degree of matching between the local data distributions of each participant. This allows for the selection of a subset of participants with similar data distributions and reliable data quality to participate in gradient aggregation. This mechanism effectively mitigates the performance degradation of the global model caused by low-quality data or severe data distribution shifts (such as data poisoning attacks), thereby ensuring the training stability and detection accuracy of the financial anomaly detection model under the federated learning framework.

[0035] The entire multidimensional statistical feature extraction process is executed independently on each participant's local machine, without sharing raw data with other participants, fully complying with the data privacy protection principles of federated learning. The construction of the empirical distribution histogram and the moment estimation operation are both completed based on the local dataset. The computational complexity is linearly related to the amount of local data, demonstrating good scalability in practical deployments and supporting the real-time feature extraction needs of large-scale financial transaction data.

[0036] The aggregation server determines a subset of participants for aggregation based on the degree of matching between the data distribution characteristics of each participant and performs homomorphic addition synthesis, including: For each participant's data distribution characteristics, the distribution mean, distribution variance, and distribution skewness of each transaction feature dimension are compared pairwise across participants. The difference in distribution mean, distribution variance, and distribution skewness between each pair of participants in each transaction feature dimension are calculated. The absolute values ​​of the above three differences are summed to generate a multidimensional distribution difference measure between participants. Perform cross-participant difference comparison calculation on the distribution divergence index of each participant, and use the absolute value of the difference as the distribution divergence difference measure value among participants. Then, perform a fusion calculation on the multidimensional distribution difference measure value and the distribution divergence difference measure value to generate a comprehensive distribution difference value among participants. Based on the comprehensive distribution difference value among the participants, the participants are clustered according to the connectivity relationship between the comprehensive distribution difference values, and the union of the participants belonging to the same cluster is determined as the participant subset. Within the ciphertext domain, a homomorphic addition synthesis operation is performed on the encrypted gradient data of each participant in the participant subset. The encrypted gradient data of all participants in the participant subset are accumulated element by element by the ciphertext domain homomorphic addition to generate global encrypted gradient data.

[0037] After receiving the data distribution characteristics uploaded by each participant, the aggregation server first needs to quantitatively evaluate the similarity of the data distributions among the participants to determine which participants are suitable for this round of gradient aggregation. The core of this process lies in characterizing the distribution differences among the participants from multiple statistical dimensions and synthesizing these differences into a comparable scalar value for subsequent clustering.

[0038] Let the total number of participants be The index set of the participants is Any two participating parties and (in , The multidimensional distribution difference measure between the two is calculated as follows. For each transaction feature dimension... ( ), take the participants respectively With participating parties Distribution mean in this dimension and Distribution variance and skewness of distribution and Calculate the absolute differences of the three types of statistics, and add them together to obtain the one-dimensional distribution difference value on that dimension. , expressed as: For all The summation of the single-dimensional distribution differences of each transaction feature dimension yields the participants' information. With participating parties Multidimensional distribution difference measure between : The smaller the value, the closer the statistical characteristics of the data distribution of the two participants in each transaction feature dimension are, and the more suitable they are to participate in gradient aggregation together in the same aggregation round.

[0039] After completing the multidimensional distribution difference measurement, it is necessary to further examine the differences in the overall dispersion of the data among the various participants. When uploading data distribution characteristics, each participant simultaneously uploaded the distribution dispersion index of their local data. This indicator comprehensively reflects the degree of dispersion of data across various transaction characteristic dimensions. For participants... With participating parties In each transaction feature dimension The absolute value of the difference between the two distribution divergence indices is calculated, and then summed over all dimensions to obtain the distribution divergence difference measure. : in Indicates the participating parties In the Distribution divergence index across each transaction feature dimension Indicates the participating parties The distribution divergence index is used in the corresponding dimension. The introduction of the distribution divergence index enables the aggregation server not only to determine whether the data distribution centers and shapes of two participants are similar, but also to identify participants whose data is extremely sparse or extremely concentrated, thereby avoiding unreasonable bias effects on global gradient aggregation.

[0040] The multidimensional distribution difference measure and the distribution divergence difference measure are linearly fused to generate the participants. With participating parties The overall distribution difference value between them : in and To integrate the weighting coefficients, satisfy the following conditions: , , In actual deployment, and The value can be configured according to the complexity of data distribution in specific financial business scenarios. For example, when the data dispersion of each participant differs significantly, it can be appropriately increased. The proportion of [a factor] is adjusted to enhance sensitivity to differences in dispersion. The overall distribution difference value is [calculated / calculated]. This forms a symmetric matrix, where each element characterizes the overall difference in data distribution between the corresponding participating pairs.

[0041] Based on the aforementioned comprehensive distribution difference value matrix, clustering is performed on each participant. Specifically, a distribution difference threshold is set. If two parties and satisfy If so, then a connection is considered to exist between the two, meaning that in a graph structure where the participants are nodes and the connection relationships are edges, and There is an edge between them. Performing connected component analysis on this graph structure groups all participants belonging to the same connected component into the same cluster. Threshold The threshold setting needs to be reasonably configured based on the business characteristics of financial transaction data. Too small a threshold will result in overly scattered clusters and too few participants in the aggregation, reducing the generalization ability of the global model; too large a threshold will force participants with significantly different data distributions into the same cluster, introducing noisy gradients and affecting the convergence quality of the anomaly detection model. In practice, the model convergence speed on the validation set can be used as a feedback signal. Make adaptive adjustments.

[0042] After clustering is completed, the largest cluster (i.e., the cluster containing the most participants) is selected as the subset of participants for this round of aggregation. If multiple largest clusters of the same size exist, the mean of the comprehensive distribution differences within each cluster can be compared, and the cluster with the most consistent internal distribution (smallest mean) can be selected as the subset of participants. This selection strategy ensures that the participants in the aggregation have high homogeneity in data distribution, thereby enabling the aggregated global gradient to truly reflect the statistical regularity of mainstream financial transaction data and improving the adaptability of the anomaly detection model to the local environment of each participant.

[0043] After determining the subset of participants, a homomorphic addition synthesis operation is performed on the encrypted gradient data of each participant in the subset within the ciphertext domain. The encrypted gradient data uploaded by each participant is a ciphertext vector resulting from homomorphic encryption of their local parameter gradient data. Each position in the ciphertext vector corresponds to the ciphertext of the gradient component of a trainable parameter in the local anomaly detection parameter set. The aggregation server does not need to hold the homomorphic encryption private key; it directly performs element-wise ciphertext accumulation on the encrypted gradient data of all participants in the subset within the ciphertext domain. Let there be a total of [number missing] participants in the subset. Each participant's encrypted gradient data ciphertext vector is denoted as follows: Globally encrypted gradient data We obtain this through element-wise homomorphic addition: in This represents the addition operation of the ciphertext field as defined by the homomorphic encryption scheme. Since homomorphic encryption schemes satisfy the additive homomorphism, the result of the ciphertext field accumulation, after decryption, is equivalent to the element-wise summation of the plaintext gradient data of each participant. This allows for the aggregation of global gradients without exposing the local plaintext gradients of any participant. The aggregation server will then generate the global encrypted gradient data. The data is distributed to each participant, who then uses a homomorphic encryption private key to decrypt it and obtain the decrypted global gradient data. This data is used to update the local anomaly detection parameter set and drive the continuous iterative optimization of the federated anomaly detection model.

[0044] The determination of the subset of participants in the aggregation also includes updating the subset of participants during iteration and distributing the results to each participant, including: In each round of iterative training, the comprehensive distribution difference value among the participants is recalculated based on the data distribution characteristics uploaded by each participant in this round, and the range of participants included in the participant subset is updated based on the recalculated comprehensive distribution difference value among the participants. The global encrypted gradient data and the participant identification information of the updated participant subset are jointly encapsulated into an aggregation result data packet, and the aggregation result data packet is distributed to each participant; After receiving the aggregation result data packet, each participating party parses the participating party identification information carried in the aggregation result data packet, compares its own participating party identification with the participating party identification information, and determines whether it belongs to the participating party subset. For participants whose participant identifiers belong to the subset of participants, homomorphic decryption is performed on the global encrypted gradient data using the homomorphic encryption private key to obtain decrypted gradient data, and the parameter values ​​in the local anomaly detection parameter set are updated based on the decrypted gradient data. For participants whose participant identifier does not belong to the subset of participants, the parameter values ​​in the current local anomaly detection parameter set are kept unchanged, and the anomaly determination operation is performed using the unchanged local anomaly detection parameter set.

[0045] At the start of each iteration of training, each participant extracts the latest data distribution features based on its local financial transaction dataset and uploads them to the aggregation server. Upon receiving these uploaded features, the aggregation server does not reuse the previously calculated comprehensive distribution difference value. Instead, it re-executes the calculation process for the comprehensive distribution difference value between participants based on the latest uploaded data distribution features. The core motivation for this design is that the distribution characteristics of financial transaction data drift over time due to changes in the market environment or adjustments to business rules. If the initial distribution features are used for clustering, the composition of the participant subsets will fail to reflect the dynamic evolution of the data distribution among participants, introducing aggregation bias and weakening the generalization ability of the global model. By recalculating the comprehensive distribution difference value in each iteration, the aggregation server can perceive the real-time changes in the data distribution of each participant and dynamically adjust the composition range of the participant subsets participating in the aggregation, ensuring that the gradient data fused in each aggregation comes from a group of participants with similar distribution characteristics.

[0046] After recalculating the overall distribution difference value, the aggregation server updates the participant subset based on the calculation results of this round, determining the set of participants for this round of aggregation. The update process of the participant subset maintains the same judgment logic as the initial partitioning process, that is, based on the comparison between the overall distribution difference value and the preset distribution difference threshold, participants whose overall distribution difference value is lower than the threshold are grouped into the same subset. If a participant was in the participant subset in the previous round, but in this round, its data distribution has shifted significantly, causing its overall distribution difference value to exceed the threshold, then that participant will be removed from the participant subset in this round; conversely, if a participant was not included in the participant subset in the previous round, but in this round, its data distribution is closer to the distribution of the participants in the subset, then that participant can be re-included in the subset. This dynamic adjustment mechanism gives the aggregation framework the ability to adapt to data distribution drift, effectively avoiding the aggregation quality degradation problem caused by the solidification of participant subsets.

[0047] After updating the participant subset, the aggregation server encapsulates the global encrypted gradient data generated by homomorphic addition synthesis in the encrypted domain during this round, along with the participant identification information of the updated participant subset, into an aggregation result data packet. The participant identification information is stored as a structured list in a specific field of the data packet, with each identifier corresponding to a unique identifier of a participant belonging to the participant subset in this round. The global encrypted gradient data is stored as a ciphertext vector in another field of the data packet, with each component of the ciphertext vector corresponding to the encrypted gradient aggregation result of each trainable parameter in the local anomaly detection parameter set. After encapsulation, the aggregation result data packet is broadcast and distributed by the aggregation server to all participants, regardless of whether they belong to the participant subset in this round.

[0048] Upon receiving the aggregation result data packet, each participant first parses the packet, retrieving the complete list of participant identifiers for the current round's subset from the participant identifier information field. Then, it compares its own participant identifier against this list item by item to determine if it appears, thus classifying itself as a participant subset for the current round. This comparison operation is performed independently in each participant's local environment, eliminating the need for additional communication with the aggregation server. This saves communication overhead and avoids additional exposure of participant identity information on the network. The comparison result directly determines the type of operation that the participant performs on its local anomaly detection parameter set in this round.

[0049] For participants whose comparison results indicate they belong to a subset of participants, they use their held homomorphic encryption private key to perform homomorphic decryption on the ciphertext vector of the global encrypted gradient data extracted from the aggregation result data packet, restoring the ciphertext vector to the plaintext gradient vector, thus obtaining the decrypted gradient data. Each gradient component in the decrypted gradient data corresponds to the global aggregated gradient estimate of each trainable parameter in the local anomaly detection parameter set. Based on the decrypted gradient data, the participants update the values ​​of each parameter in the local anomaly detection parameter set according to a preset parameter update rule. After the update, the local anomaly detection parameter set reflects the cross-participant gradient information fused in this round of aggregation, possessing stronger anomaly detection capabilities.

[0050] For participants whose comparison results indicate they do not belong to a subset of participants, all parameter values ​​in the current local anomaly detection parameter set remain unchanged, and no update operation based on the globally encrypted gradient data is performed on any parameter. The rationale for this strategy is that not belonging to a subset of participants means that the participant's data distribution differs significantly from the group of participants in this round of aggregation. Forcing the use of aggregation gradients oriented towards other distribution groups to update local parameters would cause the parameter values ​​in the local anomaly detection parameter set to deviate from the optimal direction of the local data distribution, thus reducing anomaly detection accuracy. By keeping the parameters unchanged, the participant retains the parameter state formed after the previous round of updates and uses the local anomaly detection parameter set in this state to continue performing anomaly judgment operations on the financial transaction data to be detected, generating anomaly judgment results, and triggering an alarm information push process when an anomaly is detected.

[0051] This differentiated parameter update strategy ensures the stability of local models for participants with heterogeneous data while maintaining the training efficiency of the overall federated learning framework. Participants not belonging to a subset are not completely excluded from the system; they continue to upload the latest data distribution characteristics in each iteration, allowing the aggregation server to continuously monitor the changing trends of their distribution characteristics. Once a participant's data distribution in subsequent rounds approaches the distribution of participants within the subset, the participant can be reintegrated into the subset to participate in gradient aggregation and parameter updates in subsequent rounds. This dynamic inclusion and exclusion mechanism enables the system to adaptively manage the composition of participant subsets during long-term operation, balancing aggregation quality and participant coverage, effectively supporting the continuous and efficient execution of multi-participant collaborative anomaly detection tasks in financial big data scenarios.

[0052] The system performs anomaly detection calculations on the financial transaction data to be inspected and generates alarm information, including: Based on the updated local anomaly detection parameter set, multidimensional feature transformation operations are performed on each transaction feature dimension of the financial transaction data to be detected to generate a dimensionality-reduced feature representation. Feature reconstruction operations are then performed on the dimensionality-reduced feature representation to obtain reconstructed transaction data. The reconstructed transaction data has the same feature structure as the financial transaction data to be detected. The deviation magnitude between the original values ​​of each transaction feature dimension of the financial transaction data to be detected and the reconstructed values ​​of the corresponding dimensions in the reconstructed transaction data is calculated, and the reconstruction deviation components of each dimension are generated. The reconstruction deviation components of each dimension are fused according to the contribution coefficients of each dimension determined in the parameter training stage to obtain the anomaly judgment result. The anomaly determination result is compared with the preset anomaly determination benchmark. When the anomaly determination result exceeds the preset anomaly determination benchmark, the financial transaction data to be detected is marked as abnormal financial transaction data, and the target transaction feature dimension with the largest reconstructed deviation component value is determined as the anomaly attribution dimension. For transaction records marked as abnormal financial transaction data, the abnormal attribution dimension and transaction attribute information of the transaction record are extracted, and the abnormal attribution dimension and transaction attribute information are encapsulated into alarm information and pushed to the corresponding participants.

[0053] After completing federated training and updating the local anomaly detection parameter set, the first step in performing anomaly detection operations on the financial transaction data to be detected is to perform multi-dimensional feature transformation on each of its transaction feature dimensions. Specifically, the original feature vector of the financial transaction data to be detected is input into an encoder structure defined by the local anomaly detection parameter set. This encoder performs a combination of linear projection and nonlinear activation transformation on each input transaction feature dimension, mapping the high-dimensional original feature space to a low-dimensional latent representation space, thereby generating a dimensionality-reduced feature representation. The dimensionality of the dimensionality-reduced feature representation is much lower than that of the original feature space, retaining the main structural information of normal financial transaction data, while its ability to express anomaly patterns is constrained by the bottleneck structure. This characteristic is the core basis for subsequent reconstruction bias calculation.

[0054] After obtaining the dimensionality-reduced feature representation, it is input into a decoder structure defined by the local anomaly detection parameter set. The decoder performs feature reconstruction operations on the dimensionality-reduced feature representation, mapping the low-dimensional latent representation layer by layer back to a feature structure with the same dimension as the original financial transaction data to be detected, generating reconstructed transaction data. Each transaction feature dimension of the reconstructed transaction data corresponds to the reconstructed estimate of the corresponding dimension in the original financial transaction data to be detected. For data that conforms to normal transaction patterns, the encoder-decoder structure can complete the reconstruction with relatively small information loss, and the deviation between the reconstructed value and the original value is small. For abnormal transaction data that deviates from the normal pattern, since the encoder has difficulty effectively representing its abnormal structure in low-dimensional space, the reconstructed value of the decoder will produce a significant deviation from the original value.

[0055] After obtaining the reconstructed transaction data, the deviation magnitude between the original values ​​and the reconstructed values ​​is calculated for each transaction feature dimension, generating reconstruction deviation components for each dimension. Let the financial transaction data to be detected be the first... The original values ​​for each transaction feature dimension are: The corresponding reconstruction value is Then the first Reconstruction bias components in each dimension Defined as ,in This represents absolute value operations. Reconstruction deviation components for each dimension. Together they form a reconstruction bias vector, which reflects the degree to which the financial transaction data to be detected deviates from the normal pattern in each dimension of transaction features.

[0056] The reconstruction bias components of each dimension are not fused in an equally weighted manner, but rather based on the contribution coefficients of each dimension determined during the parameter training phase. Weighted fusion is performed to obtain the final anomaly detection result. Contribution coefficient During the training phase, the contribution of each transaction feature dimension to the overall reconstruction error is determined by analyzing its degree of contribution. Dimensions that exhibit significant natural fluctuations in normal trading have relatively low contribution coefficients to avoid false alarms triggered by normal fluctuations. Dimensions that remain stable in normal trading but show significant deviations during anomalies have relatively high contribution coefficients, thereby improving the sensitivity to detect true anomalies. The weighted fusion operation can be expressed as... ,in The total number of transaction feature dimensions. Satisfy normalization constraints .

[0057] Anomaly detection results Compared with the preset anomaly judgment criteria Comparison. Preset anomaly detection criteria. During the parameter training phase, the reconstruction error distribution is statistically determined based on normal trading data. Typically, the high quantile value of the reconstruction error distribution based on normal trading data is taken to ensure that the false positive rate on normal trading data remains within an acceptable range. When, the financial transaction data to be detected is marked as abnormal financial transaction data; when If the data is deemed normal, no alarm is triggered. For cases marked as abnormal, the transaction feature dimension corresponding to the largest value in the reconstructed deviation vector is further identified, and this dimension is determined as the anomaly attribution dimension. The anomaly attribution dimension points to the single feature dimension that contributed most significantly to the deviation in this anomaly determination, providing an interpretable basis for anomaly localization for subsequent manual review and risk handling.

[0058] For transaction records marked as abnormal financial transaction data, the anomaly attribution dimension information and transaction attribute information of the transaction record are extracted. Transaction attribute information includes, but is not limited to, structured fields such as transaction timestamp, transaction amount, counterparty identifier, transaction channel type, and transaction geographical location. These fields are directly read from the original data structure of the transaction record without feature transformation processing to preserve the readability of its original semantics. The name of the anomaly attribution dimension, along with the corresponding reconstructed deviation component value and transaction attribute information, are encapsulated into structured alarm information, which is then pushed to the risk management terminal of the corresponding participant through a pre-defined message push channel.

[0059] The structured encapsulation design of alarm information enables participating parties to quickly pinpoint the specific characteristics of anomalies. For example, when the anomaly attribution dimension is "single transaction amount," participating parties can directly focus on the abnormal transaction amount for manual review, without needing to check every field of the entire transaction record. This anomaly attribution mechanism based on reconstructed deviation components significantly improves the interpretability of alarm information and the efficiency of actual business processing compared to the black-box judgment method that only outputs anomaly scores. Furthermore, the reconstructed deviation component value itself is also pushed as part of the alarm information, allowing participating parties to quantitatively assess the degree of anomaly, thereby rationally allocating manual review resources and prioritizing high-risk abnormal transaction records with the largest deviation amplitude.

[0060] In real-world deployment scenarios, the aforementioned anomaly detection and alarm generation processes are executed sequentially on each incoming financial transaction data in a streaming manner, ensuring real-time anomaly detection. Each time the local anomaly detection parameter set undergoes a new round of federated updates, a preset anomaly detection benchmark is established. The system is then recalibrated based on the reconstruction error distribution of the new parameter set on the validation dataset to ensure that the anomaly judgment benchmark is consistent with the expressive power of the current parameter set and to avoid the judgment benchmark shift caused by parameter updates.

[0061] A second aspect of this invention provides an AI-driven real-time anomaly detection system for financial big data, comprising: The local training unit is used by each participant to construct a local anomaly detection parameter set based on the local financial transaction dataset, and to perform iterative training on the local anomaly detection parameter set using the local financial transaction dataset to generate local parameter gradient data. An encrypted upload unit is used by each participant to perform homomorphic encryption operation on the local parameter gradient data using a homomorphic encryption public key to generate encrypted gradient data, extract data distribution features based on the local financial transaction dataset, and upload the encrypted gradient data and the data distribution features to the aggregation server. The aggregation and distribution unit is used by the aggregation server to determine a subset of participants in the aggregation based on the matching degree between the data distribution characteristics of each participant, to perform homomorphic addition on the encrypted gradient data of each participant in the subset of participants in the ciphertext domain to synthesize global encrypted gradient data, and to distribute the global encrypted gradient data to each participant. The decryption and update unit is used by each participant to perform homomorphic decryption operation on the received global encrypted gradient data using the homomorphic encryption private key to obtain decryption gradient data, and update the local anomaly detection parameter set based on the decryption gradient data; The anomaly alarm unit is used by each participant to perform anomaly judgment calculations on the financial transaction data to be detected based on the updated local anomaly detection parameter set, and generate anomaly judgment results. When the anomaly judgment result indicates an anomaly, an alarm message is generated and pushed to the corresponding participant.

[0062] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0063] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0064] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AI-driven real-time anomaly detection method for financial big data, characterized in that, include: Each participant constructs a local anomaly detection parameter set based on the local financial transaction dataset, and performs iterative training on the local anomaly detection parameter set using the local financial transaction dataset to generate local parameter gradient data. Each participant uses a homomorphic encryption public key to perform homomorphic encryption operations on the local parameter gradient data to generate encrypted gradient data. Based on the local financial transaction dataset, data distribution features are extracted, and the encrypted gradient data and the data distribution features are uploaded to the aggregation server. The aggregation server determines a subset of participants in the aggregation based on the degree of matching between the data distribution characteristics of each participant. Within the ciphertext domain, it performs homomorphic addition on the encrypted gradient data of each participant in the subset to synthesize global encrypted gradient data, and then distributes the global encrypted gradient data to each participant. Each participant uses a homomorphic encryption private key to perform homomorphic decryption on the received global encryption gradient data to obtain decryption gradient data, and updates the local anomaly detection parameter set based on the decryption gradient data; Each participant performs anomaly detection calculations on the financial transaction data to be detected based on the updated local anomaly detection parameter set to generate anomaly detection results. When the anomaly detection results indicate an anomaly, an alarm message is generated and pushed to the corresponding participant.

2. The method according to claim 1, characterized in that, Generating the encrypted gradient data and extracting the data distribution features includes: Gradient encoding operation is performed on the gradient components corresponding to each trainable parameter in the local parameter gradient data, and each gradient component is mapped to the corresponding coefficient position in the plaintext polynomial coefficient space of the fully homomorphic encryption scheme to obtain encoded gradient data. The homomorphic encryption public key is used to perform a fully homomorphic encryption operation on the encoded gradient data to generate encrypted gradient data. The fully homomorphic encryption operation supports homomorphic addition and homomorphic multiplication operations within the ciphertext domain. Multidimensional statistical feature extraction operations are performed on the numerical distribution of each transaction feature dimension related to the anomaly detection task in the local financial transaction dataset to generate the data distribution features. The data distribution features include the distribution mean, distribution variance, distribution skewness of each transaction feature dimension, and the dispersion index of each transaction feature dimension in a preset quantile interval. The encrypted gradient data and the data distribution features are respectively encapsulated in a data structure format. The encrypted gradient data is generated by adding participant identifiers and iteration round identifiers to the encrypted gradient data, and the data distribution features are generated by adding participant identifiers and iteration round identifiers to the data distribution features. The encrypted gradient data and the distribution feature data are then uploaded to the aggregation server via independent secure transmission channels.

3. The method according to claim 2, characterized in that, Performing the multidimensional statistical feature extraction operation includes: Nonparametric probability density estimation is performed on the numerical values ​​of each transaction feature dimension. A preset number of equal-width quantile intervals are divided within the value range of each transaction feature dimension. The frequency of samples falling into each interval is counted. The sample frequency of each interval is divided by the total number of samples in that dimension to obtain the sample frequency of each interval, and an empirical distribution histogram of each transaction feature dimension is generated. Based on the empirical distribution histogram, a moment estimation operation is performed to calculate the first-order raw moment of each transaction feature dimension as the distribution mean, the second-order central moment of each transaction feature dimension as the distribution variance, and the third-order central moment of each transaction feature dimension, after standardization, as the distribution skewness. The frequency statistics of each transaction feature dimension in each quantile interval are compared with the preset global benchmark frequency distribution. The cumulative result of the absolute value of the frequency difference in each interval is weighted according to the sample frequency of the interval as the dispersion index. The distribution mean, distribution variance, distribution skewness, and dispersion index are structured and organized in the form of multidimensional feature vectors to form the data distribution features. The data distribution features and the encoded gradient data are then synchronously uploaded to the aggregation server.

4. The method according to claim 1, characterized in that, The aggregation server determines a subset of participants for aggregation based on the degree of matching between the data distribution characteristics of each participant and performs homomorphic addition synthesis, including: For each participant's data distribution characteristics, the distribution mean, distribution variance, and distribution skewness of each transaction feature dimension are compared pairwise across participants. The difference in distribution mean, distribution variance, and distribution skewness between each pair of participants in each transaction feature dimension are calculated. The absolute values ​​of the above three differences are summed to generate a multidimensional distribution difference measure between participants. Perform cross-participant difference comparison calculation on the distribution divergence index of each participant, and use the absolute value of the difference as the distribution divergence difference measure value among participants. Then, perform a fusion calculation on the multidimensional distribution difference measure value and the distribution divergence difference measure value to generate a comprehensive distribution difference value among participants. Based on the comprehensive distribution difference value among the participants, the participants are clustered according to the connectivity relationship between the comprehensive distribution difference values, and the union of the participants belonging to the same cluster is determined as the participant subset. Within the ciphertext domain, a homomorphic addition synthesis operation is performed on the encrypted gradient data of each participant in the participant subset. The encrypted gradient data of all participants in the participant subset are accumulated element by element by the ciphertext domain homomorphic addition to generate global encrypted gradient data.

5. The method according to claim 4, characterized in that, The determination of the subset of participants in the aggregation also includes updating the subset of participants during iteration and distributing the results to each participant, including: In each round of iterative training, the comprehensive distribution difference value among the participants is recalculated based on the data distribution characteristics uploaded by each participant in this round, and the range of participants included in the participant subset is updated based on the recalculated comprehensive distribution difference value among the participants. The global encrypted gradient data and the participant identification information of the updated participant subset are jointly encapsulated into an aggregation result data packet, and the aggregation result data packet is distributed to each participant; After receiving the aggregation result data packet, each participating party parses the participating party identification information carried in the aggregation result data packet, compares its own participating party identification with the participating party identification information, and determines whether it belongs to the participating party subset. For participants whose participant identifiers belong to the subset of participants, homomorphic decryption is performed on the global encrypted gradient data using the homomorphic encryption private key to obtain decrypted gradient data, and the parameter values ​​in the local anomaly detection parameter set are updated based on the decrypted gradient data. For participants whose participant identifier does not belong to the subset of participants, the parameter values ​​in the current local anomaly detection parameter set are kept unchanged, and the anomaly determination operation is performed using the unchanged local anomaly detection parameter set.

6. The method according to claim 1, characterized in that, The system performs anomaly detection calculations on the financial transaction data to be inspected and generates alarm information, including: Based on the updated local anomaly detection parameter set, multidimensional feature transformation operations are performed on each transaction feature dimension of the financial transaction data to be detected to generate a dimensionality-reduced feature representation. Feature reconstruction operations are then performed on the dimensionality-reduced feature representation to obtain reconstructed transaction data. The reconstructed transaction data has the same feature structure as the financial transaction data to be detected. The deviation magnitude between the original values ​​of each transaction feature dimension of the financial transaction data to be detected and the reconstructed values ​​of the corresponding dimensions in the reconstructed transaction data is calculated, and the reconstruction deviation components of each dimension are generated. The reconstruction deviation components of each dimension are fused according to the contribution coefficients of each dimension determined in the parameter training stage to obtain the anomaly judgment result. The anomaly determination result is compared with the preset anomaly determination benchmark. When the anomaly determination result exceeds the preset anomaly determination benchmark, the financial transaction data to be detected is marked as abnormal financial transaction data, and the target transaction feature dimension with the largest reconstructed deviation component value is determined as the anomaly attribution dimension. For transaction records marked as abnormal financial transaction data, the abnormal attribution dimension and transaction attribute information of the transaction record are extracted, and the abnormal attribution dimension and transaction attribute information are encapsulated into alarm information and pushed to the corresponding participants.

7. An AI-driven real-time anomaly detection system for financial big data, used to implement the method as described in any one of claims 1-6, characterized in that, include: The local training unit is used by each participant to construct a local anomaly detection parameter set based on the local financial transaction dataset, and to perform iterative training on the local anomaly detection parameter set using the local financial transaction dataset to generate local parameter gradient data. An encrypted upload unit is used by each participant to perform homomorphic encryption operation on the local parameter gradient data using a homomorphic encryption public key to generate encrypted gradient data, extract data distribution features based on the local financial transaction dataset, and upload the encrypted gradient data and the data distribution features to the aggregation server. The aggregation and distribution unit is used by the aggregation server to determine a subset of participants in the aggregation based on the matching degree between the data distribution characteristics of each participant, to perform homomorphic addition on the encrypted gradient data of each participant in the subset of participants in the ciphertext domain to synthesize global encrypted gradient data, and to distribute the global encrypted gradient data to each participant. The decryption and update unit is used by each participant to perform homomorphic decryption operation on the received global encrypted gradient data using the homomorphic encryption private key to obtain decryption gradient data, and update the local anomaly detection parameter set based on the decryption gradient data; The anomaly alarm unit is used by each participant to perform anomaly judgment calculations on the financial transaction data to be detected based on the updated local anomaly detection parameter set, and generate anomaly judgment results. When the anomaly judgment result indicates an anomaly, an alarm message is generated and pushed to the corresponding participant.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images