Financial big data scene-oriented time series data adaptive compression system

Abstract

The invention discloses a financial big data scene-oriented time series data adaptive compression system, and relates to the technical field of data compression. The invention aims to solve the problems of high I / O (Input / Output) load, low caching efficiency and low throughput caused by low compression rate, large reconstruction error and incapability of reflecting the condition of a financial system in the conventional data compression method. The system comprises K edge node ends and a cloud end, each edge node end is used for encoding a timestamp of the original data sequence, then utilizing a compression algorithm recommendation network to obtain a recommendation data compression algorithm, adopting the recommendation data compression algorithm to perform data compression on numerical values of the original data sequence to obtain a compressed original data sequence, and meanwhile, obtaining compression algorithm recommendation network parameters; sending the compression algorithm recommendation network parameters to the cloud; and the cloud side uses a compression algorithm to recommend network parameters to judge whether the edge node is an abnormal edge node, and if the edge node is the abnormal edge node, the current edge node is frozen. The method is used for data compression in the financial system.

Description

Technical Field

[0001] This invention relates to the field of data compression technology, and in particular to an adaptive compression system for time-series data in financial big data scenarios. Background Technology

[0002] Financial data is a collection of data used in the financial field to describe various information such as financial markets, financial institutions, and financial instruments. Financial data is generated extremely rapidly; transaction data is updated every second. High-frequency trading systems generate tens of thousands of records per second (such as NYSE market data). Each transaction record contains a nanosecond-level timestamp, meaning that regardless of the actual value of the timestamp, it occupies the same amount of storage space. During continuous bidding, timestamps grow strictly linearly, meaning the difference between each timestamp and the previous one is fixed. However, existing storage methods still store every complete timestamp, resulting in wasted storage space. Therefore, experts in this field have proposed data compression methods to address this problem.

[0003] Current time-series data compression methods mainly include Delta-of-Delta encoding compression, Zstandard compression, static compression, and compression models based on historical data. Delta-of-Delta encoding compression is based on the principle that the timestamp difference (delta) is usually much smaller than the length of the timestamp itself, combined with variable-length encoding to reduce the storage space required for data. However, financial trading systems often experience sudden market events (circuit breakers, block trades), which can cause abrupt changes in the distribution of timestamp differences, resulting in a 41%-58% reduction in the compression ratio of Delta-of-Delta encoding compression. Therefore, Delta-of-Delta encoding compression suffers from low compression ratios when facing sudden market events, leading to poor I / O performance. Zstandard compression aims to provide a high compression ratio while maintaining high decompression speed. However, current financial data storage does not distinguish between order book data and daily candlestick chart data. When Zstandard compression is used uniformly, critical details of the order book are lost, disrupting the original sequence correlation, resulting in low data compression ratios, large reconstruction errors, and ultimately high hardware I / O load and low cache efficiency. Static compression methods involve compressing files at creation time and using the compressed data throughout its lifecycle. However, data in financial trading systems fluctuates dramatically. Static compression strategies can lead to large reconstruction errors exceeding risk control thresholds, resulting in data distortion, high hardware load, amplified disk read / write operations, and cache mismatch, ultimately causing low overall I / O throughput. Historical data-based compression models utilize machine learning techniques to predict and compress time-series data. However, financial trading data is influenced by social factors, causing predictions based on historical data to fail to reflect the current situation, thus rendering data compression strategies ineffective. Summary of the Invention

[0004] This invention addresses the problems of compression strategy failure, high I / O load, low caching efficiency, and low I / O throughput in existing data compression methods, and proposes a time-series data adaptive compression system for financial big data scenarios.

[0005] An adaptive compression system for time-series data in financial big data scenarios, comprising: K edge nodes and the cloud;

[0006] Each edge node is used to encode the timestamp of the original data sequence provided by the exchange, and then uses the compression algorithm recommendation network to obtain the recommended data compression algorithm. The recommended data compression algorithm is used to compress the values ​​of the original data sequence to obtain the compressed original data sequence. At the same time, the compression algorithm recommendation network parameters are obtained and sent to the cloud.

[0007] The original time-series data includes: price data sequence, transaction volume data sequence, order direction data sequence, and corresponding nanosecond-level timestamps;

[0008] The cloud platform uses the compression algorithm corresponding to the current edge node to recommend network parameters to determine whether the current edge node is an abnormal edge node. If it is an abnormal edge node, the current edge node is frozen; otherwise, no processing is performed.

[0009] Furthermore, each edge node includes: a data preprocessing module, a real-time feature extraction module, an intelligent feature analysis and decision-making module, an adaptive compression strategy matching module, a timestamp dynamic encoding and data compression module, and a compression algorithm recommendation network parameter acquisition module;

[0010] The data preprocessing module is used to obtain the raw time-series data provided by the exchange and preprocess the raw time-series data to obtain the preprocessed raw time-series data.

[0011] The preprocessing includes: outlier removal, missing value filling, and clock synchronization;

[0012] The clock synchronization includes: time zone calibration and leap second compensation;

[0013] The real-time feature extraction module is used to obtain multi-dimensional feature sequences using preprocessed raw time-series data;

[0014] The multidimensional feature sequences include: price difference sequence, trading volume difference sequence, and order direction binary sequence;

[0015] The intelligent feature analysis and decision-making module obtains the data scenario and key indicators corresponding to the multi-dimensional feature sequence, and sends the data scenario to the adaptive compression strategy matching module and the key indicators to the timestamp dynamic encoding and data compression module.

[0016] The adaptive compression strategy matching module uses the data scenario corresponding to the multi-dimensional feature sequence to obtain the set of candidate compression algorithms corresponding to each feature sequence.

[0017] The timestamp dynamic encoding and data compression module is used to encode the timestamps of the original sequence, and recommend compression algorithms from the set of candidate compression algorithms using key indicators and compression algorithm recommendation network. The recommended compression algorithm is then used to compress the numerical part of the original sequence to obtain the compressed result of the original sequence.

[0018] The compression algorithm recommendation network parameter acquisition module is used to acquire the current parameters of the compression algorithm recommendation network and send the current parameters of the compression algorithm recommendation network to the cloud.

[0019] Furthermore, the data preprocessing module is used to acquire the raw time-series data provided by the exchange and preprocess the raw time-series data to obtain preprocessed raw time-series data, specifically as follows:

[0020] First, obtain the abnormal timestamps of the original time series data and delete them;

[0021] Then, the missing timestamp positions are filled with the average of the timestamp before and after the missing timestamp, and the missing value positions in the original time series data are filled with the average of the values ​​before and after the missing value.

[0022] Finally, time zone calibration and leap second compensation are performed on the original time series data to obtain the preprocessed original time series data.

[0023] Furthermore, the real-time feature extraction module is used to obtain a multi-dimensional feature sequence using the preprocessed original time-series data, specifically as follows:

[0024]

[0025] in, It is a price data series. It is a price difference sequence. It is a sequence of transaction volume data. It is a multidimensional feature sequence. It is a volume difference sequence. It is an order direction sequence. It is a binary sequence indicating the order direction.

[0026] Furthermore, the intelligent feature analysis and decision-making module utilizes the data scenarios and key indicators corresponding to the multi-dimensional feature sequences, specifically as follows:

[0027] The price difference sequence, volume difference sequence, order direction binary sequence, and preset rules are used to determine the market event identifier and data type label corresponding to the current raw data;

[0028] The data scenarios include: market event identifiers and data type tags;

[0029] The market event tags include: opening, circuit breaker, audit traceability, trend analysis, and normal trading.

[0030] The data type tags include: market data, candlestick data, and risk data;

[0031] Key indicators are obtained using price difference sequences and volume difference sequences, specifically:

[0032] The key indicators include: market volatility and liquidity indicators;

[0033]

[0034]

[0035]

[0036]

[0037] in, It's market volatility. It is price market volatility. It is the market volatility of trading volume. It is a price difference sequence. It is a volume difference sequence. It is the maximum value in the price difference sequence. It is the minimum value in the price difference sequence. It is the total number of data points in the price difference sequence. It is the maximum value in the volume difference sequence. It is the minimum value in the volume difference sequence. It is the total number of data points in the volume difference sequence. It is a liquidity indicator. It is in the price difference series Time value and Time value difference, It is in the price data series Value corresponding to time, It is in the price data series Value corresponding to time, It is in the original transaction volume sequence The value corresponding to the given time.

[0038] Furthermore, the adaptive compression strategy matching module obtains a set of candidate compression algorithms for each feature sequence using the data scenario corresponding to the multi-dimensional feature sequence, specifically:

[0039] If the market event is labeled as "opening" and the data type is labeled as "order book data," then a lossless compression algorithm will be used.

[0040] The lossless compression algorithms include: OrderBook-Delta, FIX-Message, and Depth-Spread-ANS;

[0041] If the market event is labeled as "opening" and the data type is labeled as "K-line data", then a controllable lossy compression algorithm will be used.

[0042] The controllable lossy compression algorithms include: Risk-Compress, Volatility-ANS, ZFP-Hybrid, and KLine-Segment;

[0043] If the market event is labeled "opening" and the data type is labeled "risk data", then a controllable lossy compression algorithm will be used.

[0044] If the market event is labeled "circuit breaker" and the data type is labeled "market data," then a lossless compression algorithm will be used.

[0045] If the market event is labeled "circuit breaker" and the data type is labeled "risk data," then a lossless compression algorithm will be used.

[0046] If the market event is tagged as audit traceability and the data type is tagged as order book data, candlestick data, or risk data, then a lossless compression algorithm will be used.

[0047] If the market event is tagged as trend analysis and the data type is tagged as order book data, then a lossless compression algorithm will be used.

[0048] If the market event is tagged as trend analysis and the data type is tagged as candlestick data, then a lossy compression algorithm will be used.

[0049] The lossy compression algorithms include: KLine-Segment, KLine-Fit, and Volatility-ANS;

[0050] If the market event is labeled as trend analysis and the data type is labeled as risk data, then a controllable lossy compression algorithm will be used.

[0051] If the market event is labeled as normal trading and the data type is labeled as order book data, then a lossless compression algorithm will be used.

[0052] If the market event is labeled as normal trading and the data type is labeled as candlestick data, then a controllable lossy compression algorithm is used.

[0053] If the market event is labeled as normal trading and the data type is labeled as risk data, then a controllable lossy compression algorithm is used.

[0054] Furthermore, the timestamp dynamic encoding and data compression module includes: a timestamp encoding unit and a numerical part compression unit;

[0055] The timestamp encoding unit is used to encode the timestamps of the original sequence to obtain the encoded timestamps, specifically as follows:

[0056] Determine whether the market event identifier corresponding to the current original sequence is a circuit breaker. If it is a circuit breaker, execute step a; otherwise, execute step b.

[0057] a. Encode the timestamp into a triplet Specifically:

[0058] First, set the timestamp dynamic offset number. ;

[0059] Then, using Get and Specifically:

[0060]

[0061]

[0062] in, It is the difference in timestamps before and after the circuit breaker is triggered. It is the dynamic offset of the timestamp. It is the floor function, m is the floor function. Number the blocks by size. yes The remainder;

[0063] Storage triples The total number of bits is optimized to [a certain value] using a data decomposition and data encoding method. ;

[0064] in, It is the maximum value of the dynamic displacement;

[0065] b. Select market event timestamps and perform third-order differential encoding based on the market event timestamps, specifically:

[0066]

[0067]

[0068] in, It is a third-order differential encoding of timestamps. It is a second-order differential encoding of timestamps. yes The second-order differential encoding of the time corresponding to the timestamp. It is a first-order differential encoding of timestamps. It is the selected market event timestamp. It is a moment Event timestamp;

[0069] The numerical compression unit uses key indicators and a compression algorithm recommendation network to obtain a recommended compression algorithm, and uses the recommended compression algorithm to compress the numerical part of the original sequence to obtain the compression result.

[0070] Furthermore, the numerical compression unit utilizes key indicators and a compression algorithm recommendation network to obtain a recommended compression algorithm, and uses the recommended compression algorithm to compress the numerical portion of the original sequence to obtain the compression result, specifically as follows:

[0071] A1. Obtain the compression algorithm recommendation network, specifically:

[0072] A1.1 Construct the financial space state, action space, and reward function;

[0073] First, the financial space status is constructed based on market volatility and liquidity indicators as follows:

[0074]

[0075]

[0076] in, It is the rate of change in trading volume. , It is the TPU utilization rate. , It's memory usage. This is the length of the order queue; 0 indicates an empty queue, and 1 indicates a full queue. It is a preset maximum reconstruction error threshold. This is the preset upper limit for data processing latency. It was a moment. yes Current state;

[0077] Then, each compression algorithm in the set of candidate compression algorithms is combined with each configuration in the set of preset configuration parameters, and the resulting set of all combinations is the workspace;

[0078] Then, construct the reward function, specifically:

[0079]

[0080]

[0081]

[0082]

[0083]

[0084] in, It is the maximum value in the current original sequence. It is the minimum value in the current original sequence. It is the data at time t in the current original sequence. It is the data at time t in the sequence obtained after the compressed original sequence is reconstructed. It is the total number of times in the current original sequence. It is the reward obtained by performing the q-th action at time t;

[0085] A1.2 Construct a generative adversarial network (GAN) and perform a pre-training cycle on the GAN. Use the pre-trained GAN as the recommendation network for the compression algorithm.

[0086] A2. Based on the financial space state, action space, and reward function, a compression algorithm is used to recommend the network online to obtain the recommendation compression algorithm. The original sequence is then compressed using the recommendation compression algorithm to obtain the compressed original sequence.

[0087] Furthermore, the generative adversarial network includes: a generator and a discriminator;

[0088] The generator recommends compression algorithms from a set of candidate compression algorithms using the financial space state.

[0089] The network parameters of the generator are updated in the following way:

[0090]

[0091]

[0092]

[0093]

[0094]

[0095] in, It is the first Parameters of the generator network during the first round of training. It is the first Generator network parameters trained in +1 round. yes +1 round of training learning rate; It is an expectation. It is an economic return weighting factor. It is the actual volatility. It is historical volatility. It is a parameter gradient, It is an economic reward. It is the average of the reward function value sequence. It is the standard deviation of the reward function value sequence. It is a pre-set risk-free return;

[0096] The discriminator is used to execute the recommendation compression algorithm and obtain the reward function value, and then send the reward function value to the generator.

[0097] Furthermore, the cloud platform uses the compression algorithm corresponding to the current edge node to recommend network parameters to determine whether the current edge node is an abnormal edge node, specifically:

[0098] B1. Obtain the compression algorithm recommendation network parameters in the edge nodes corresponding to each exchange to obtain the global network parameters, specifically:

[0099]

[0100]

[0101]

[0102] in, This represents the data value of exchange k, where K is the total number of edge nodes. It is a sequence of reward function values The average value, It is a preset risk-free return. It is a profit series standard deviation The compression algorithm recommends network parameters in edge node k. , It is the momentum factor. It is the change in the exponential moving average. , It is the change in the exponential moving average at the previous time point. It is the time difference value;

[0103] B2. Use global network parameters to determine if edge nodes are abnormal, specifically: If If the condition is met, it indicates that edge node k is abnormal; otherwise, it indicates that edge node k is normal.

[0104] in, It is the standard deviation of global network parameters.

[0105] The beneficial effects of this invention are as follows:

[0106] This invention proposes an adaptive compression system for time-series data in financial big data scenarios. Taking raw high-frequency trading time-series data as input, it performs clock synchronization and preprocessing on cross-market data. Then, real-time feature extraction is performed at edge nodes to calculate multi-dimensional features such as price difference sequences, trading volume difference sequences, and order direction data sequences. The system then obtains the data scenarios and key indicators corresponding to these multi-dimensional feature sequences. Based on the data scenarios, this invention determines the range of compression algorithms to be selected and uses a compression algorithm recommendation network to obtain recommended compression algorithms, thereby achieving high-frequency trading data compression. This invention applies different compression algorithms to data with different features without destroying the correlation between sequences, thus improving the data compression ratio, reducing reconstruction errors, and ultimately reducing hardware I / O load and improving cache efficiency. This invention constructs a compression algorithm recommendation network based on the features and data states of high-frequency trading data. The compression algorithm recommendation network can provide corresponding rewards based on the effectiveness of the compression algorithm. This invention can recommend the optimal compression algorithm, reducing reconstruction errors in data compression and preventing reconstruction errors from exceeding risk thresholds, thereby avoiding data distortion after compression, reducing hardware load, and ultimately avoiding disk read / write amplification and cache mismatch, thus improving overall I / O throughput. This invention considers the characteristics of current data in financial systems and recommends data compression strategies that are more suitable for data compression in financial systems. It accurately reflects the current data situation and makes the data compression algorithm more effective. Simultaneously, this invention forms a federated optimization structure between cloud and edge nodes and can detect abnormal edge nodes, ensuring the security of the financial data system. Attached Figure Description

[0107] Figure 1 This is a diagram of the overall architecture of the present invention;

[0108] Figure 2 This is a diagram of a dual-adversarial network structure.

[0109] Figure 3 This is a flowchart of the system modules. Detailed Implementation

[0110] Specific implementation method one: as follows Figure 1 and Figure 3 As shown, the time-series data adaptive compression system for financial big data scenarios in this embodiment includes: K edge nodes and the cloud;

[0111] Each edge node is used to encode the timestamp of the original data sequence provided by the exchange, and then uses the compression algorithm recommendation network to obtain the recommended data compression algorithm. The recommended data compression algorithm is used to compress the values ​​of the original data sequence to obtain the compressed original data sequence. At the same time, the compression algorithm recommendation network parameters are obtained and sent to the cloud.

[0112] The original time-series data includes: price data sequence, transaction volume data sequence, order direction data sequence, and corresponding nanosecond-level timestamps;

[0113] The cloud platform uses the compression algorithm corresponding to the current edge node to recommend network parameters to determine whether the current edge node is an abnormal edge node. If it is an abnormal edge node, the current edge node is frozen; otherwise, no processing is performed.

[0114] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that each edge node includes: a data preprocessing module, a real-time feature extraction module, an intelligent feature analysis and decision-making module, an adaptive compression strategy matching module, a timestamp dynamic encoding and data compression module, and a compression algorithm recommendation network parameter acquisition module;

[0115] The data preprocessing module is used to acquire raw time-series data provided by multiple exchanges and preprocess the raw time-series data to obtain preprocessed raw time-series data.

[0116] The original time-series data includes: price data sequence, transaction volume data sequence, order direction data sequence, and nanosecond-level timestamps corresponding to each sequence;

[0117] The preprocessing includes: outlier removal, missing value filling, and clock synchronization;

[0118] The clock synchronization includes: time zone calibration and leap second compensation;

[0119] The real-time feature extraction module is used to obtain multi-dimensional feature sequences using preprocessed raw time-series data;

[0120] The multidimensional feature sequences include: price difference sequence (unit: bps), trading volume difference sequence (unit: million USD), and order direction binary sequence;

[0121] The intelligent feature analysis and decision-making module obtains the data scenario and key indicators corresponding to the multi-dimensional feature sequence, and sends the data scenario to the adaptive compression strategy matching module and the key indicators to the timestamp dynamic encoding and data compression module.

[0122] The adaptive compression strategy matching module uses the data scenario corresponding to the multi-dimensional feature sequence to obtain the set of candidate compression algorithms corresponding to each feature sequence.

[0123] The timestamp dynamic encoding and data compression module is used to encode the timestamps of the original sequence, and recommend compression algorithms from the set of candidate compression algorithms using key indicators and compression algorithm recommendation network parameters. The recommended compression algorithm is then used to compress the numerical part of the original sequence to obtain the compressed result of the original sequence.

[0124] The compression algorithm recommendation network parameter acquisition module is used to acquire the current parameters of the compression algorithm recommendation network and send the current parameters of the compression algorithm recommendation network to the cloud.

[0125] Other steps or parameters are the same as in Specific Implementation Method 1.

[0126] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that: the data preprocessing module is used to acquire raw time-series data provided by multiple exchanges, and preprocesses the raw time-series data to obtain preprocessed raw time-series data, specifically as follows:

[0127] First, obtain the abnormal timestamps of the original time series data and delete them;

[0128] Then, the missing timestamp positions are filled with the average of the timestamp before and after the missing timestamp, and the missing value positions in the original time series data are filled with the average of the values ​​before and after the missing value.

[0129] Finally, time zone calibration and leap second compensation are performed on the original time series data to obtain the preprocessed original time series data.

[0130] In existing global multi-exchange data fusion scenarios, time zone conversion (such as the time difference between NYSE and LSE) and leap second correction features are not considered, resulting in accumulated cross-market time series alignment errors of 0.1-2.3 seconds, affecting the timing consistency of arbitrage strategies. The data preprocessing and clock synchronization module in this embodiment uses unified time series data after time zone calibration and leap second compensation to ensure cross-market data consistency; it performs clock synchronization processing on the input cross-market time series data and outputs a standardized timestamp sequence, providing a unified benchmark for subsequent feature extraction and compression.

[0131] Other steps or parameters are the same as in specific implementation method one or two.

[0132] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that the real-time feature extraction module is used to obtain multi-dimensional feature sequences using the preprocessed original time-series data, specifically as follows:

[0133]

[0134] in, It is a price data series. It is a price difference sequence. It is a sequence of transaction volume data. , It is a multidimensional feature sequence. It is a volume difference sequence. It is an order direction sequence. It is a binary sequence indicating the order direction;

[0135] This module processes time-series data in real time at edge nodes, extracting core features such as price, volume, and order flow. Price difference is obtained by calculating the difference between adjacent bid or ask prices; volume difference is calculated by the difference in volume between adjacent time points; order direction is represented by buy / sell binary codes. The extracted feature sequences are sent to the feature analysis module for scene recognition and compression strategy decisions.

[0136] The other steps and parameters are the same as in any of the specific implementation methods one to three.

[0137] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the intelligent feature analysis and decision-making module utilizes the data scenarios and key indicators corresponding to the multi-dimensional feature sequences, specifically:

[0138] The price difference sequence, volume difference sequence, order direction binary sequence, and preset rules are used to determine the market event identifier and data type label corresponding to the current raw data;

[0139] The data scenarios include: market event identifiers and data type tags;

[0140] The market event tags include: opening, circuit breaker, audit traceability, trend analysis, and normal trading.

[0141] The data type tags include: market data, candlestick data, and risk data;

[0142] The preset rule is as follows:

[0143] Opening: When the trading day changes, if the system detects that the timestamp falls within the opening time range announced by the exchange, and the price difference sequence and the volume difference both show an increase greater than the preset range within the window, the "opening" event is triggered.

[0144] The window length is a preset evaluation time, set to 60 seconds;

[0145] Circuit Breaker: When the cumulative price difference exceeds the preset regulatory threshold (e.g., 5% or 500bps) within N consecutive sliding windows, and the order direction sequence continues to have a one-sided proportion greater than the preset percentage (buy or sell ratio exceeding 95%), while the trading volume decreases by the preset percentage value (below 20% of the stable period), the system determines that the "circuit breaker" is triggered and generates a circuit breaker event identifier.

[0146] Audit Traceability: When data channel metadata or compliance templates declare strong traceability (traceability=Strong, retention ≥ N years, loss_mode=Lossless), and the original stream is an unaggregated order book / order stream (containing forensically verifiable fields such as order_id and seq), and CRC / hash check bits are detected to be enabled within the window, while the three features exhibit transaction-by-transaction granularity (the volume difference sequence and price difference sequence contain a greater than preset percentage of difference data less than the preset difference threshold, and the order direction binary sequence alternates continuously rather than aggregates), the system marks this data as "Audit Traceability".

[0147] When the proportion of positive numbers in the price difference sequence is greater than Or the proportion of negative numbers is greater than If the increase in trading volume remains stable within the normal range, a "trend analysis" event identifier will be generated.

[0148] The normal trading band is defined as P25–P75 within a fixed window (e.g., 60 seconds) based on the ΔVolume distribution of the same time point over the past 20 trading days. If historical data is insufficient, the most recent 5 days are used temporarily. This band is updated daily and used for identifying "normal trading" and comparing it with thresholds.

[0149] If no opening, circuit breaker, audit tracing, or trend analysis events are triggered, and the price difference, volume difference, and order direction binary sequence within the sliding window all fall within their respective near M-day steady-state ranges (e.g., within the IQR range, buy / sell ratio of 40%–60%, and cumulative fluctuation of a single window <100bps), and the order book updates at a regular pace, the system will mark the data as "normal trading".

[0150] The normal rhythm refers to whether the trading frequency is normal. It is the median trading frequency of "the same time in the past 20 trading days" med_freq. If the trading frequency of the current 60-second window falls within [0.75×med_freq, 1.25×med_freq], then it is a normal rhythm.

[0151] Key indicators are obtained using price difference sequences and volume difference sequences, specifically:

[0152] The key indicators include: market volatility and liquidity indicators;

[0153]

[0154]

[0155]

[0156]

[0157] in, It's market volatility. It is price market volatility. It is the market volatility of trading volume. It is a price difference sequence. It is a volume difference sequence. It is the maximum value in the price difference sequence. It is the minimum value in the price difference sequence. It is the total number of data points in the price difference sequence. It is the maximum value in the volume difference sequence. It is the minimum value in the volume difference sequence. It is the total number of data points in the volume difference sequence. It is a liquidity indicator. It is in the price difference series Time value and Time value difference, It is in the price data series Value corresponding to time, It is in the price data series Value corresponding to time, It is in the original transaction volume sequence The value corresponding to the given time.

[0158] This implementation combines multidimensional feature sequences with preset rules for evaluation. The system performs real-time analysis on indicators such as price volatility, implied volatility, order book depth, and trading frequency to determine whether the current market is experiencing severe volatility or a sudden event, and classifies different data streams. For example, the system can identify extreme market conditions by detecting abnormal changes in implied volatility and trading volume, or distinguish high-frequency order book data from low-frequency trend data based on order book depth. The analysis results are output as structured scenario information, which will serve as input for compression strategy matching.

[0159] The other steps and parameters are the same as in any of the specific implementation methods one to four.

[0160] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that: the adaptive compression strategy matching module utilizes the data scenario corresponding to the multi-dimensional feature sequence to obtain the set of candidate compression algorithms corresponding to each feature sequence, specifically:

[0161] The set of candidate compression algorithms for each feature sequence is obtained according to the rules shown in Table 1;

[0162] Table 1

[0163] Market Events Data types Compression levels Typical Algorithm opening Betting odds data lossless OrderBook-Delta, FIX-Message opening K-line data Controllable loss KLine-Segment (ε small) opening Risk data Controllable loss Risk-Compress (ε=f(ρ)) Circuit Breaker Betting odds data lossless OrderBook-Delta Circuit Breaker Risk data lossless Risk-Compress (ε=0) Audit traceability All types lossless OrderBook-Delta, FIX-Message, Depth-Spread-ANS Trend Analysis K-line data Damaged KLine-Segment, KLine-Fit, Volatility-ANS Trend Analysis Risk data Controllable loss Risk-Compress (ε=f(ρ)) Trend Analysis Betting odds data lossless OrderBook-Delta (ε=0) Normal trading Betting odds data lossless OrderBook-Delta (ε=0) Normal trading K-line data Controllable loss Volatility-ANS, ZFP-Hybrid Normal trading Risk data Controllable loss Risk-Compress (ε=f(ρ))

[0164] In the circuit breaker scenario, no candlestick data will be displayed;

[0165] For controllable lossy algorithms, the choice between lossy or lossless compression methods is determined based on the error threshold ε.

[0166] First, the error threshold ε is dynamically set based on the risk level, specifically as follows:

[0167] ε = f(Business Importance, Risk Level)

[0168] The importance of the business is pre-defined by the financial data system, specifically as follows: Audit traceability scenario + market data = high importance; Real-time monitoring + risk data = medium importance; Trend analysis + candlestick data = low importance; Real-time monitoring + market data = high importance; Real-time monitoring + market data = medium importance; Trend analysis + risk data = medium importance; Audit traceability + market data = high importance; Audit traceability + candlestick data = high importance; Audit traceability + risk data = high importance.

[0169] Calculate price volatility by weighting and market liquidity indicators The risk value R is obtained as follows:

[0170]

[0171] in, yes , The weight is set to 0.5 by default, and R is the risk value;

[0172] The risk level score R is between 0 and 1. This is considered high-risk. Then it is considered medium risk, if Low risk.

[0173] If the risk level is high, the output ε = 0; if the business importance is high within the risk level, the output ε = 0.05. Price volatility; in the risk level, the output for business importance is ε=0.1. Price volatility; low risk level, medium business importance, output ε=0.2 Price volatility; low risk level, high business importance, output ε=0.1 Price volatility, low risk level, business importance, output ε=0.2 Price volatility ε indicates a low risk level and low business importance; ε = 0.3 Price volatility;

[0174] Then, when Lossless compression is performed in real time, completely preserving the original appearance of the risky data; when Controlled lossy compression is enabled at certain times to truncate small residuals and improve the compression ratio.

[0175] Lossy compression algorithms include: Volatility-ANS, Event-Driven-Segment, and Predictive-LSTM-Codec; lossless compression algorithms include: OrderBook-Delta, FIX-Message, Tick-Stream, Correlation-Delta, Depth-Spread-ANS, and Entropy-Adaptive-Huffman; the controllable lossy compression algorithms include: Risk-Compress, Volatility-ANS, ZFP-Hybrid, and KLine-Segment.

[0176] The configuration parameters and corresponding levels of the compression algorithm are as follows:

[0177] The data compression algorithms are shown in Table 1.

[0178] Algorithm Name type computational complexity Core Principles Financial expansion characteristics OrderBook-Delta lossless O(1) Tier price difference differential + ZigZag code Dynamic gear depth detection FIX-Message lossless O(n) Field-level dictionary compression (Tag value mapping optimization) Supports FIX 5.0 SP2 extension Volatility-ANS Damaged O(n) Asymmetric number system + volatility truncation (retaining 3 decimal places) Greek value sensitivity analysis embedding Tick-Stream lossless O(n) Compression of the transaction direction flag (packing 0 / 1 bits) Supports time-sharing transaction aggregation KLine-Segment Damaged O(n) Piecewise linear fitting (maximum deviation ≤ 0.1%) Automatic labeling of trend inflection points Risk-Compress mix O(n) Monte Carlo path selective sampling (preserving 99% confidence interval) Risk exposure metadata binding Correlation-Delta lossless O(n) Price difference / ratio difference between adjacent assets + cross-entropy compression Dynamic correlation detection with automatic switching of difference dimensions Event-Driven-Segment Damaged O(n) Piecewise linear fitting based on event-triggered boundaries Key nodes are retained based on event priority, while other segments can be coarsened. Depth-Spread-ANS lossless O(n) Asymmetric number system + adaptive ANS entropy coding Adjust residual cutoff threshold according to liquidity level ZFP-Hybrid Damaged O(n) ZFP algorithm (block-level floating-point compression) + multi-stage interpolation The error threshold can be finely adjusted for European / American option surfaces respectively. Predictive-LSTM-Codec Damaged O(n) LSTM prediction + residual coding Online updates to model weights adapt to seasonal and cyclical changes. Entropy-Adaptive-Huffman lossless O(n) Dynamically construct the Huffman tree and update the coding table online. The dictionary is automatically adjusted based on the frequency of message fields.

[0179] This implementation maps scenario classification results to specific compression schemes. The system has a built-in scalable financial compression algorithm library, including twelve lightweight compression algorithms designed for different data types. Combining intelligent layered compression protocols and financial regulatory requirements, the system determines the appropriate algorithm combination and compression level for different data usage scenarios.

[0180] The other steps and parameters are the same as those in any of the specific implementation methods one to five.

[0181] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One to Six in that the timestamp dynamic encoding and data compression module includes: a timestamp encoding unit and a numerical part compression unit;

[0182] The timestamp encoding unit is used to encode the timestamps of the original sequence to obtain the encoded timestamps, specifically as follows:

[0183] Determine whether the market event identifier corresponding to the current original sequence is a circuit breaker. If it is a circuit breaker, execute step a; otherwise, execute step b.

[0184] a. Encode the timestamp into a triplet Specifically:

[0185] First, set the timestamp dynamic offset number. ;

[0186] Then, using Get and Specifically:

[0187]

[0188]

[0189] in, It is the difference in timestamps before and after the circuit breaker is triggered. It is the dynamic offset of the timestamp (suitable for events in the range of 10ns to 1ms). It is the floor function, m is the floor function. Number the blocks by size. yes The remainder represents a small offset falling within the current m-th block, used to refine the representation of the error range;

[0190] Storage triples The total number of bits is optimized by using data decomposition-based data encoding. ;

[0191] in, It is the maximum value of the dynamic displacement;

[0192] b. Select market event timestamps and perform third-order differential encoding based on the market event timestamps, specifically:

[0193]

[0194]

[0195] in, It is a third-order differential encoding of timestamps. It is a second-order differential encoding of timestamps. yes The second-order differential encoding of the time corresponding to the timestamp. It is a first-order differential encoding of timestamps. It is the selected market event timestamp. It is the event timestamp at the current moment;

[0196] The numerical compression unit uses key indicators and a compression algorithm recommendation network to obtain a recommended compression algorithm, and then uses the recommended compression algorithm to compress the numerical part of the original sequence to obtain the compression result, specifically:

[0197] A1. Obtain the compression algorithm recommendation network, specifically:

[0198] A1.1 Construct the financial space state, action space, and reward function;

[0199] First, the financial space status is constructed based on market volatility and liquidity indicators as follows:

[0200]

[0201]

[0202] in, It is the rate of change in trading volume. , It is the TPU (Tensor Processing Unit) utilization rate. , It's memory usage. This is the length of the order queue; 0 indicates an empty queue, and 1 indicates a full queue. It is a preset maximum reconstruction error threshold. This is the preset upper limit for data processing latency. It was a moment. yes Current state;

[0203] Then, each compression algorithm in the set of candidate compression algorithms is combined with each configuration in the set of preset configuration parameters, and the resulting set of all combinations is the workspace;

[0204] Then, construct the reward function, specifically:

[0205]

[0206]

[0207]

[0208]

[0209]

[0210] in, It is the maximum value in the current original sequence. It is the minimum value in the current original sequence. It is the data at time t in the current original sequence. It is the data at time t in the sequence obtained after the compressed original sequence is reconstructed. It is the total number of times in the current original sequence. It is the reward obtained by performing the q-th action at time t;

[0211] The number of compliance checks passed is obtained as follows: compliance checks are performed separately on reconstruction error, compression algorithm selection, and data compression time. The number of items that pass the compliance check is the number of compliance checks passed. The total number of compliance checks is 3.

[0212] The refactoring error compliance check is as follows: whether the refactoring error is less than the set error threshold. If the refactoring error is less than the preset error threshold, the compliance check is passed; otherwise, the compliance check is failed.

[0213] The compliance check for compression algorithm selection is as follows: whether the system selects the recommended compression algorithm. If the recommended compression algorithm is selected, the compliance check is passed; otherwise, the compliance check is failed.

[0214] The data compression time compliance check is as follows: whether the data compression time is less than the preset time threshold. If the data compression time is less than the preset time threshold, the compliance check is passed; otherwise, the compliance check is failed.

[0215] In this step, the compression ratio, regulatory score, and reconstruction error all range from 0 to 1, with a liquidity penalty. .

[0216] A1.2, such as Figure 2 As shown, a generative adversarial network (GAN) is constructed and pre-trained for 100 rounds. The pre-trained GAN is then used as the recommendation network for the compression algorithm.

[0217] The generative adversarial network includes: a generator and a discriminator;

[0218] The generator recommends compression algorithms from a set of candidate compression algorithms using the financial space state.

[0219] The network parameters of the generator are obtained by updating the reward function value, specifically:

[0220]

[0221]

[0222]

[0223]

[0224]

[0225] in, It is the first Parameters of the generator network during the first round of training. It is the first Generator network parameters trained in +1 round. yes +1 training round learning rate, initial value 0.001; It is an expectation. It is an economic return weighting factor. It is the actual volatility. It is historical volatility. It is a parameter gradient, It is an economic reward. It is the average of the reward function value sequence. It is the standard deviation of the reward function value sequence. It is a pre-set risk-free return;

[0226] The discriminator is used to execute the recommendation compression algorithm and obtain the reward function value, and then send the reward function value to the generator;

[0227] Generative adversarial networks are pre-trained using historical data (such as market data from 2008 and 2020).

[0228] A2. Based on the financial space state, action space, and reward function, a compression algorithm is used to recommend the network online to obtain the recommendation compression algorithm. The original sequence is then compressed using the recommendation compression algorithm to obtain the compressed original sequence.

[0229] The other steps and parameters are the same as those in any of the specific implementation methods one to six.

[0230] This implementation utilizes dynamic timestamp encoding technology to encode nanosecond-level timestamps of high-frequency trading time series using variable-length bit-width encoding, reducing storage redundancy. The system selects key time points (such as daily opening or market event trigger times) as encoding anchors, calculates third-order differences for subsequent timestamp differences, and employs variable-length encoding. In the event of a sudden event (such as a circuit breaker trigger), adaptive shift encoding compresses the timestamp differences into triples (k,m,r), recording only significant bit segments, thus significantly reducing the required bit width. After completing timestamp encoding, the module compresses the data value portion according to the algorithm selected in step 4. For example, the OrderBook-Delta algorithm is used to compress order book price spreads, or the Volatility-ANS algorithm is used to approximately truncate implied volatility data. This invention establishes a market adaptive mechanism through deep reinforcement learning algorithms to continuously optimize the compression strategy. Specifically, a dual-network adversarial learning framework is adopted, where the generator outputs candidate compression strategies, and the discriminator evaluates the compliance and quality of the compression results. Both are trained adversarially to improve strategy performance. The system is designed with a state space incorporating variables such as market volatility, liquidity indicators, trading volume change rate, and computational resource utilization, as well as a reward function targeting metrics such as compression ratio, compliance score, reconstruction error, and liquidity penalty. The training process consists of two phases: offline pre-training and real-time online training. In the pre-training phase, the model is initialized using historical extreme market data; in the online phase, network parameters are continuously updated based on real-time market feedback. The system can dynamically adjust the compression algorithm selection and parameter settings according to the current data conditions, maximizing compression efficiency and data quality while strictly meeting compliance requirements.

[0231] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One through Seven in that: the cloud utilizes the compression algorithm corresponding to the current edge node to recommend network parameters to determine whether the current edge node is an abnormal edge node, specifically as follows:

[0232] B1. Obtain the compression algorithm recommendation network parameters in the edge nodes corresponding to each exchange to obtain the global network parameters, specifically:

[0233]

[0234]

[0235]

[0236] in, This represents the data value of exchange k, where K is the total number of edge nodes. It is a sequence of reward function values The average value, It is a preset risk-free return. It is a constant that takes a local minimum or 0. It is a profit series standard deviation The compression algorithm recommends network parameters in edge node k. , It is the momentum factor. It is the change in the exponential moving average. , It is the change in the exponential moving average at the previous time point. It is the instantaneous difference of parameter updates at the current moment.

[0237] Reward function value sequence The reward function value sequence is obtained as follows: using historical data as a validation set in the cloud, the network is recommended using the compression algorithm corresponding to the validation edge node k, and the obtained reward function value sequence is used as the profit sequence.

[0238] B2. Use global network parameters to determine if edge nodes are abnormal, specifically: If If the condition is met, it indicates that edge node k is abnormal; otherwise, it indicates that edge node k is normal.

[0239] in, It is the standard deviation of global network parameters.

[0240] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

Claims

1. A time-series data adaptive compression system for financial big data scenarios, characterized by: The system includes: K edge nodes and a cloud platform; Each edge node is used to encode the timestamp of the original data sequence provided by the exchange, and then uses the compression algorithm recommendation network to obtain the recommended data compression algorithm. The recommended data compression algorithm is used to compress the values ​​of the original data sequence to obtain the compressed original data sequence. At the same time, the compression algorithm recommendation network parameters are obtained and sent to the cloud. The original time-series data includes: price data sequence, transaction volume data sequence, order direction data sequence, and corresponding nanosecond-level timestamps; The cloud platform uses the compression algorithm corresponding to the current edge node to recommend network parameters to determine whether the current edge node is an abnormal edge node. If it is an abnormal edge node, the current edge node is frozen; otherwise, no processing is performed.

2. The adaptive compression system for time-series data in financial big data scenarios according to claim 1, characterized in that: Each edge node includes: a data preprocessing module, a real-time feature extraction module, an intelligent feature analysis and decision-making module, an adaptive compression strategy matching module, a timestamp dynamic encoding and data compression module, and a compression algorithm recommendation network parameter acquisition module; The data preprocessing module is used to obtain the raw time-series data provided by the exchange and preprocess the raw time-series data to obtain the preprocessed raw time-series data. The preprocessing includes: outlier removal, missing value filling, and clock synchronization; The clock synchronization includes: time zone calibration and leap second compensation; The real-time feature extraction module is used to obtain multi-dimensional feature sequences using preprocessed raw time-series data; The multidimensional feature sequences include: price difference sequence, trading volume difference sequence, and order direction binary sequence; The intelligent feature analysis and decision-making module obtains the data scenario and key indicators corresponding to the multi-dimensional feature sequence, and sends the data scenario to the adaptive compression strategy matching module and the key indicators to the timestamp dynamic encoding and data compression module. The adaptive compression strategy matching module uses the data scenario corresponding to the multi-dimensional feature sequence to obtain the set of candidate compression algorithms corresponding to each feature sequence. The timestamp dynamic encoding and data compression module is used to encode the timestamps of the original sequence, and recommend compression algorithms from the set of candidate compression algorithms using key indicators and compression algorithm recommendation network. The recommended compression algorithm is then used to compress the numerical part of the original sequence to obtain the compressed result of the original sequence. The compression algorithm recommendation network parameter acquisition module is used to acquire the current parameters of the compression algorithm recommendation network and send the current parameters of the compression algorithm recommendation network to the cloud.

3. The adaptive compression system for time-series data in financial big data scenarios according to claim 2, characterized in that: The data preprocessing module is used to acquire the raw time-series data provided by the exchange and preprocess the raw time-series data to obtain preprocessed raw time-series data, specifically as follows: First, obtain the abnormal timestamps of the original time series data and delete them; Then, the missing timestamp positions are filled with the average of the timestamp before and after the missing timestamp, and the missing value positions in the original time series data are filled with the average of the values ​​before and after the missing value. Finally, time zone calibration and leap second compensation are performed on the original time series data to obtain the preprocessed original time series data.

4. The adaptive compression system for time-series data in financial big data scenarios according to claim 3, characterized in that: The real-time feature extraction module is used to obtain multi-dimensional feature sequences from the preprocessed raw time-series data, specifically: in, It is a price data series. It is a price difference sequence. It is a sequence of transaction volume data. It is a multidimensional feature sequence. It is a volume difference sequence. It is an order direction sequence. It is a binary sequence indicating the order direction.

5. The adaptive compression system for time-series data in financial big data scenarios according to claim 4, characterized in that: The intelligent feature analysis and decision-making module utilizes the data scenarios and key indicators corresponding to the multi-dimensional feature sequences, specifically: The price difference sequence, volume difference sequence, order direction binary sequence, and preset rules are used to determine the market event identifier and data type label corresponding to the current raw data; The data scenarios include: market event identifiers and data type tags; The market event tags include: opening, circuit breaker, audit traceability, trend analysis, and normal trading. The data type tags include: market data, candlestick data, and risk data; Key indicators are obtained using price difference sequences and volume difference sequences, specifically: The key indicators include: market volatility and liquidity indicators; in, It's market volatility. It is price market volatility. It is the market volatility of trading volume. It is a price difference sequence. It is a volume difference sequence. It is the maximum value in the price difference sequence. It is the minimum value in the price difference sequence. It is the total number of data points in the price difference sequence. It is the maximum value in the volume difference sequence. It is the minimum value in the volume difference sequence. It is the total number of data points in the volume difference sequence. It is a liquidity indicator. It is in the price difference series Time value and Time value difference, It is in the price data series Time-corresponding value It is in the price data series Value corresponding to time, It is in the original transaction volume sequence The value corresponding to the given time.

6. The adaptive compression system for time-series data in financial big data scenarios according to claim 5, characterized in that: The adaptive compression strategy matching module obtains a set of candidate compression algorithms for each feature sequence by utilizing the data scenario corresponding to the multi-dimensional feature sequence, specifically: If the market event is labeled as "opening" and the data type is labeled as "order book data," then a lossless compression algorithm will be used. The lossless compression algorithms include: OrderBook-Delta, FIX-Message, and Depth-Spread-ANS; If the market event is labeled as "opening" and the data type is labeled as "K-line data", then a controllable lossy compression algorithm will be used. The controllable lossy compression algorithms include: Risk-Compress, Volatility-ANS, ZFP-Hybrid, and KLine-Segment; If the market event is labeled "opening" and the data type is labeled "risk data", then a controllable lossy compression algorithm will be used. If the market event is labeled "circuit breaker" and the data type is labeled "market data," then a lossless compression algorithm will be used. If the market event is labeled "circuit breaker" and the data type is labeled "risk data," then a lossless compression algorithm will be used. If the market event is tagged as audit traceability and the data type is tagged as order book data, candlestick data, or risk data, then a lossless compression algorithm will be used. If the market event is tagged as trend analysis and the data type is tagged as order book data, then a lossless compression algorithm will be used. If the market event is tagged as trend analysis and the data type is tagged as candlestick data, then a lossy compression algorithm will be used. The lossy compression algorithms include: KLine-Segment, KLine-Fit, and Volatility-ANS; If the market event is labeled as trend analysis and the data type is labeled as risk data, then a controllable lossy compression algorithm will be used. If the market event is labeled as normal trading and the data type is labeled as order book data, then a lossless compression algorithm will be used. If the market event is labeled as normal trading and the data type is labeled as candlestick data, then a controllable lossy compression algorithm is used. If the market event is labeled as normal trading and the data type is labeled as risk data, then a controllable lossy compression algorithm is used.

7. The adaptive compression system for time-series data in financial big data scenarios according to claim 6, characterized in that: The timestamp dynamic encoding and data compression module includes: a timestamp encoding unit and a numerical part compression unit; The timestamp encoding unit is used to encode the timestamps of the original sequence to obtain the encoded timestamps, specifically as follows: Determine whether the market event identifier corresponding to the current original sequence is a circuit breaker. If it is a circuit breaker, execute step a; otherwise, execute step b. a. Encode the timestamp into a triplet Specifically: First, set the timestamp dynamic offset number. ; Then, using Get and Specifically: in, It is the difference in timestamps before and after the circuit breaker is triggered. It is the dynamic offset of the timestamp. It is the floor function, m is the floor function. Number the blocks by size. yes The remainder; Storage triples The total number of bits is optimized to [a certain value] using a data decomposition and data encoding method. ; in, It is the maximum value of the dynamic displacement; b. Select market event timestamps and perform third-order differential encoding based on the market event timestamps, specifically: in, It is a third-order differential encoding of timestamps. It is a second-order differential encoding of timestamps. yes The second-order differential encoding of the time corresponding to the timestamp. It is a first-order differential encoding of timestamps. It is the selected market event timestamp. It is a moment Event timestamp; The numerical compression unit uses key indicators and a compression algorithm recommendation network to obtain a recommended compression algorithm, and uses the recommended compression algorithm to compress the numerical part of the original sequence to obtain the compression result.

8. The adaptive compression system for time-series data in financial big data scenarios according to claim 7, characterized in that: The numerical compression unit uses key indicators and a compression algorithm recommendation network to obtain a recommended compression algorithm, and then uses the recommended compression algorithm to compress the numerical part of the original sequence to obtain the compression result, specifically: A1. Obtain the compression algorithm recommendation network, specifically: A1.1 Construct the financial space state, action space, and reward function; First, the financial space status is constructed based on market volatility and liquidity indicators as follows: in, It is the rate of change in trading volume. , It is the TPU utilization rate. , It's memory usage. This is the length of the order queue; 0 indicates an empty queue, and 1 indicates a full queue. It is a preset maximum reconstruction error threshold. This is the preset upper limit for data processing latency. It was a moment. yes Current state; Then, each compression algorithm in the set of candidate compression algorithms is combined with each configuration in the set of preset configuration parameters, and the resulting set of all combinations is the workspace; Then, construct the reward function, specifically: in, It is the maximum value in the current original sequence. It is the minimum value in the current original sequence. It is the data at time t in the current original sequence. It is the data at time t in the sequence obtained after the compressed original sequence is reconstructed. It is the total number of times in the current original sequence. It is the reward obtained by performing the q-th action at time t; A1.2 Construct a generative adversarial network (GAN) and perform a pre-training cycle on the GAN. Use the pre-trained GAN as the recommendation network for the compression algorithm. A2. Based on the financial space state, action space, and reward function, a compression algorithm is used to recommend the network online to obtain the recommendation compression algorithm. The original sequence is then compressed using the recommendation compression algorithm to obtain the compressed original sequence.

9. The adaptive compression system for time-series data in financial big data scenarios according to claim 8, characterized in that: The generative adversarial network includes: a generator and a discriminator; The generator recommends compression algorithms from a set of candidate compression algorithms using the financial space state. The network parameters of the generator are updated in the following way: in, It is the first Parameters of the generator network during the first round of training. It is the first Generator network parameters trained in +1 round. yes +1 round of training learning rate; It is an expectation. It is an economic return weighting factor. It is the actual volatility. It is historical volatility. It is a parameter gradient, It is an economic reward. It is the average of the reward function value sequence. It is the standard deviation of the reward function value sequence. It is a pre-set risk-free return; The discriminator is used to execute the recommendation compression algorithm and obtain the reward function value, and then send the reward function value to the generator.

10. The adaptive compression system for time-series data in financial big data scenarios according to claim 9, characterized in that: The cloud platform uses the compression algorithm corresponding to the current edge node to recommend network parameters to determine whether the current edge node is an abnormal edge node. Specifically: B1. Obtain the compression algorithm recommendation network parameters in the edge nodes corresponding to each exchange to obtain the global network parameters, specifically: in, This represents the data value of exchange k, where K is the total number of edge nodes. It is a sequence of reward function values The average value, It is a preset risk-free return. It is a profit series standard deviation The compression algorithm recommends network parameters in edge node k. , It is the momentum factor. It is the change in the exponential moving average. , It is the change in the exponential moving average at the previous time point. It is the time difference value; B2. Use global network parameters to determine if edge nodes are abnormal, specifically: If If the condition is met, it indicates that edge node k is abnormal; otherwise, it indicates that edge node k is normal. in, It is the standard deviation of global network parameters.

Images