Self-adaptive multi-agent stock trading method and device based on tri-state market perception

Abstract

The invention discloses a self-adaptive multi-agent stock trading method and device based on three-state market perception, and belongs to the field of artificial intelligence and financial science and technology. The method comprises the following steps: preprocessing stock historical data based on a technical analysis theory, and constructing a standardized feature system; a cattle market, a bear market and an oscillation market are divided by calculating a closing price change rate and a fluctuation rate and combining a threshold value, so that dynamic identification of market states is realized; constructing a multi-dimensional feature extraction model fusing CNN, BiGRU and an attention mechanism; a transaction environment close to a real disk is designed, a self-adaptive reward function based on a market state is constructed, a dynamic loss stopping mechanism and a bin position control strategy are integrated, and the risk control and income acquisition capability is enhanced; a multi-agent system is introduced, and a priority experience pool, a dynamic exploration strategy and a Dueling DQN architecture are adopted, so that the learning efficiency, the strategy robustness and the decision precision are improved. The method has high environment adaptability and risk response capability, and is suitable for high-frequency dynamic transaction scenes.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and fintech, and relates to an adaptive multi-agent stock trading method based on three-state market perception. Specifically, it involves constructing a dynamic trading environment that considers various market states (such as bull markets, bear markets, and sideways markets), combining fee mechanisms, open and closed position constraints, adaptive position control, and dynamic stop-loss mechanisms to simulate a trading process closer to the real financial market. Simultaneously, the system design introduces a Priority Experience Replay (PER) mechanism and a dynamic exploration strategy to construct multiple deep reinforcement learning agents with independent decision-making capabilities. An integrated voting mechanism enables collaborative decision-making among the multiple agents, thereby improving trading robustness and profitability under different market conditions. Background Technology

[0002] With the development of artificial intelligence technology, deep reinforcement learning has been widely applied in the financial field, especially in quantitative trading systems. Traditional quantitative trading strategies often rely on fixed rules and expert experience, making it difficult to adapt to the nonlinear, dynamic, and volatile nature of financial markets. In contrast, deep reinforcement learning can automatically learn optimal strategies through interaction with the market environment, and is gradually becoming a hot research area in financial intelligent trading.

[0003] (SKSahu, A. Mokhade, NDBokde, An overview of machine learning, deep learning, and reinforcement learning-based techniques in quantitative finance: recent progress and challenges) points out the challenges faced by machine learning, deep learning, and reinforcement learning in areas such as data acquisition, model interpretability, computational complexity, and financial market uncertainty. (WAAddy, AOAjayi-Nifise, BGBello, STTula, O. Odeyemi, T. Falaiye, et al., Machine learning in financial markets: A critical review of algorithmic trading and risk management) analyzes the importance of ML technology in risk management in high-frequency trading and automated trading systems. (S. Goutte, H.-V. Le, F. Liu, H.-J. Von Mettenheim, Deep learning and technical analysis in cryptocurrency market) demonstrates the effectiveness of combining deep learning models with technical analysis indicators in cryptocurrency market prediction, illustrating its performance under different market conditions. To achieve more efficient trading strategies, (Y. Deng, F. Bao, Y. Kong, Z. Ren, Q. Dai, Deep direct reinforcement learning for financial signal representation and trading) proposed a direct deep reinforcement learning (DRL) framework for financial signal representation and trading. Furthermore, (B. Yang, T. Liang, J. Xiong, C. Zhong, Deep pre-inforcement learning based on transformer and u-net framework for stock trading) proposed a stock trading DRL framework based on Transformer and U-Net structures.

[0004] However, existing reinforcement learning-based securities trading methods still face many challenges. On the one hand, real financial markets have complex market state characteristics, including various types such as bull markets, bear markets, and sideways markets, while existing methods often lack fine-grained modeling and differentiation of market states, resulting in insufficient strategy generalization ability. On the other hand, in actual trading, there are various real-world factors such as transaction fees, position limits, and capital constraints, and many reinforcement learning environments fail to effectively simulate these real trading mechanisms, causing the trained agents to perform unstablely or the strategies to fail in the real market.

[0005] Furthermore, existing single-agent systems are prone to overfitting when faced with high market volatility, unexpected events, or extreme market conditions, lacking robustness and fault tolerance. Meanwhile, traditional experience replay mechanisms treat all historical experiences equally during training, failing to prioritize high-value experiences and impacting learning efficiency and model performance. Most agents employ a fixed exploration rate, lacking the ability to perceive dynamic market risks, which can easily lead to insufficient or excessive policy exploration.

[0006] Therefore, there is an urgent need for a new stock trading method that combines market state perception capabilities, real-world trading constraints, dynamic risk control, and multi-agent collaborative learning mechanisms to improve adaptability, profitability, and system robustness in different market environments. Summary of the Invention

[0007] This invention primarily addresses the core problems of current intelligent stock trading methods, such as poor adaptability to market conditions, insufficient trading signal extraction capabilities, lagging trading decision feedback, and weak risk control measures. To address these issues, a deep reinforcement learning trading method is proposed, integrating multi-market state recognition, multi-dimensional feature extraction, and multi-agent collaborative decision-making to achieve efficient perception and robust trading in complex and dynamic market environments. In the data preparation phase, the system first collects historical stock trading data from the open market, constructs feature labels including price, volume, and technical indicators, and standardizes, normalizes, and windowes the raw data to adapt to model training and environmental input requirements. Secondly, this invention constructs a multi-market state recognition mechanism that can automatically identify typical market patterns (such as bull markets, bear markets, and sideways markets) and dynamically adjust the trading strategy structure based on the recognition results, enabling rapid adaptation and switching of the strategy to market fluctuations. Secondly, by introducing a multi-dimensional feature extractor based on convolutional neural networks (CNN), bidirectional gated recurrent units (BiGRU), and attention mechanisms, this invention deeply mines and reconstructs features from multi-source heterogeneous information such as prices, trading volumes, and technical indicators in historical stock data. This constructs a highly expressive state space, enhancing the model's ability to capture potential patterns in financial time series. In the decision learning part, this invention introduces a multi-agent architecture, combined with a priority experience replay mechanism and a dynamic exploration strategy (ε-decreasing and noise perturbation), improving sample utilization efficiency and policy search flexibility. Each agent models and learns the current market state from an independent perspective, possessing independent policy networks and feature perception capabilities. During trade execution, each agent proposes action suggestions based on its own strategy for the current state. Through a voting mechanism, the final trade decision is formed, further improving the system's robustness and generalization ability in the face of uncertainty and extreme market conditions. Ultimately, by constructing multiple complementary trading agents that work synergistically in the stock trading environment, and using multiple evaluation indicators such as cumulative return, annualized return, maximum drawdown, and Sharpe ratio to evaluate the overall performance of the trading system, the stability, effectiveness, and risk control capabilities of this multi-agent collaborative trading framework in practical applications are comprehensively measured.

[0008] The technical solution of this invention:

[0009] The adaptive multi-agent stock trading method based on three-state market perception has the following steps:

[0010] Step 100: Data crawling and preprocessing;

[0011] Step 101: Determine the data source and the area and theme of stock trading;

[0012] Among them, the data source is determined by selecting Yahoo resources for the index, futures, and individual stock datasets to be obtained;

[0013] Step 102: Use web scraping technology to obtain stock trading data from the data source;

[0014] Web crawlers, through parsing function S d =Parse(html), extract function E d =Extract(S d This is used to implement web scraping; where html is the original webpage, and S... d E represents the parsed data obtained after the parsing function Parse(·). d This represents the parsed data obtained after the analytic function Extract(·);

[0015] Step 103: Filter and clean stock trading data to remove irrelevant information and make the data cleaner and more standardized.

[0016] Data cleaning and normalization are performed using text processing techniques, denoted as E. d Given the original dataset, use the data filtering operation Filter(E) d By applying filtering rules or conditions, data that meets the requirements or standards is selected from the original data to generate an intermediate dataset F. d Next, the intermediate dataset F d Perform data cleaning operations Clean(F) d This process removes data with missing or incorrect values ​​to ensure data consistency and accuracy, ultimately generating the cleaned dataset C. d The formula is expressed as: C d =Clean(Filter(E d ); where Filter(·) represents the data filtering operation, and Clean(·) represents the data cleaning operation;

[0017] Step 104, construct feature labels;

[0018] Based on stock data such as opening price, highest price, closing price, lowest price, and trading volume, a feature label S = {O, H, L, C, V, M} is constructed for model training by calculating a moving average, exponential moving average, closing price change rate, and the closing price volatility over the past 30 days. 30 M 60 E 90 ,P r ,Ψ 30} where O is the opening price, H is the highest price, L is the lowest price, C is the closing price, V is the trading volume, and M is the lowest price. 30 It is a 30-day simple moving average, M 60 It is a 60-day simple moving average, E 90 It is the 90-day exponential moving average, P rIt is the closing price change rate, Ψ 30 It is the 30-day closing price volatility;

[0019] Use M t This represents the simple moving average over time t; N represents the time period of the moving average, indicating how many time units of price data are used to calculate the average; C t-1 This represents the closing price in the (t-1)th time unit; the simple moving average is calculated using the following formula: E t α represents the exponential moving average over time t; α represents the smoothing factor, representing the weight of the current price and the exponential moving average over the previous time unit; C t E represents the closing price in the t-th time unit. t-1 This represents the exponential moving average over a previous time unit; the formula is: E t =α·C t +(1-α)·E t-1 , P r To represent the rate of change in closing price, use C. t Represents the closing price in the t-th time unit, using C. t-1 Let represent the closing price in the (t-1)th time unit. The formula for calculating the rate of change of the closing price is: Use Ψ t Let μ represent the closing price volatility at time t, and let μ represent the mean of the closing price change rate within a sliding window of size n. The calculation formula is: Let represent the rate of change of the closing price at time step ti;

[0020] Step 105: Data standardization and windowing processing;

[0021] Stock feature data is standardized and windowed to optimize data analyzability and time series modeling performance. First, the feature labels S constructed in step 104 are linearly transformed using Min-Max standardization, calculated as follows: Where X represents the original feature matrix, X min and X max These represent the minimum and maximum values ​​for each feature dimension, respectively. This standardization method maps feature values ​​to the closed interval [0,1], effectively eliminating dimensional differences between different features, improving numerical stability, and preventing larger numerical features from dominating the model training process, thus ensuring a balanced contribution from each dimension. Furthermore, standardizing the backend data can accelerate convergence and improve the model's generalization ability. Simultaneously, to capture the short-term dependence and autocorrelation of stock data, a sliding window technique is used to construct time series samples. Each time window S... t′ The feature data, S, includes the current time t and the previous 30 time steps. t ′ =[s t-30 ,s t-29 ,…,s t-1 ], where s t-k This represents the feature vector for day tk. This windowing method displays the short-term dynamic changes of the financial time series, conforming to the Markov property assumption and providing a standardized input format for subsequent model training. In practice, the initial 30 time points are truncated to ensure consistency across all samples and avoid model training instability due to missing data. Finally, the data processed using the sliding window technique is divided into training, validation, and test sets.

[0022] Step 200: Construct a three-state market sensing mechanism;

[0023] Three market perception machines were constructed using volatility quantification analysis to dynamically identify the stock market state, providing environmental awareness capabilities for a multi-agent trading system. By calculating price volatility and combining it with threshold judgment rules, the market was divided into three states: bull market, bear market, and sideways market, thus achieving adaptive market environment awareness.

[0024] Step 201: Calculate the closing price change rate;

[0025] First, a difference operation is performed on the closing price sequence to calculate the rate of change of daily closing prices, thus measuring short-term price volatility in the market. Given a closing price sequence {C1, C2, ..., C...} t}, where t is the time step, C t This represents the closing price on day t, and the rate of change of the closing price ΔC. t The calculation formula is as follows: Where ΔC t This represents the percentage change in the closing price on day t compared to day t-1. Data from the first day (t=1) is excluded after calculation because there are no preceding values, ensuring the continuity of the data.

[0026] Step 202: Calculate the closing price slippage volatility;

[0027] Based on the closing price change rate series, volatility is calculated using a sliding window standard deviation to quantify the severity of price fluctuations. Given a window length N = 30 (i.e., 30 trading days), the volatility on day t is... in σ represents the mean rate of price change during the window period. tThis reflects the market's risk level over the past 30 trading days. Due to insufficient data for the previous N-1 time points, missing values ​​need to be removed after calculation.

[0028] Step 203: Classify market status based on thresholds;

[0029] By using a preset volatility threshold, the market state S is... t Dynamically divided into three categories: Where 0 represents a bull market, 1 represents a bear market, and 2 represents a sideways market; θ bull =2% is the bull market threshold, indicating a low-volatility market environment, usually accompanied by steady growth, θ bear =5% is the bear market threshold, indicating a highly volatile market environment, usually accompanied by sharp declines. Volatility between the two is classified as a sideways market, reflecting that the market has no clear trend.

[0030] Step 204: Output market status signal;

[0031] The three-state market perception machine quantitatively analyzes market fluctuation characteristics and ultimately outputs daily market state labels.

[0032] {State1,State2,…,State t}, where each State t ∈{0,1,2} represents the market status classification result on the t-th trading day.

[0033] Step 300: Construct a multi-dimensional feature extractor;

[0034] A multi-dimensional feature extractor is built using convolutional neural networks (CNN), bidirectional gated recurrent control units (BiGRU), and attention mechanisms for in-depth mining and analysis of financial time series.

[0035] Step 301: Use a convolutional neural network to extract feature data;

[0036] A convolutional neural network (CNN) is used to perform deep feature extraction on financial time-series features to capture local patterns and short-term dependencies in market data. This CNN employs a multi-layer convolutional architecture, using a hierarchical feature transformation mechanism to map from raw data to higher-order abstract features. Its mathematical expression is as follows: Given the time-series feature labels S∈R constructed in step 105... T×d Where T is the time step, d is the feature dimension, R is the set of real stock data, and the convolution kernel W∈R k×m Where k is the convolution width in the time dimension and m is the convolution width in the feature dimension, the convolution operation at time step i can be expressed as: Where j is the index of the convolution kernel in the time dimension, ranging from [1, k], k is the size of the convolution kernel in the time step dimension, i.e., the height of the convolution, l is the index of the convolution kernel in the feature dimension, ranging from [1, m], m is the size of the convolution kernel in the feature dimension, i.e., the width of the convolution kernel, and W j,l X represents the weights of the convolution kernel, indicating the weights of the convolution kernel at position (j,l); i+j-1,l This represents the value of the input data at position (i+j-1,l), where i is the time step index of the current convolution operation; b is the bias, added to each convolution output; Y i It is the output of the convolution operation at time step i, activated by the ReLU activation function ReLU(Y). i ) = max(0,Y i Introducing nonlinearity to generate feature map Z i Then, max pooling is used to downsample the features, with a pooling window size of p, and the output is: P i =max i≤t≤i+p-1 z t Z t Let P represent the feature map at time step t. i This is the output of the pooling result at time step i; finally, the pooled features are flattened and projected onto the target space through a fully connected layer: Among them W fc It is the weight matrix of the fully connected layer, b fc It is the bias vector of the fully connected layer. P is expanded using the flatten(·) function. i Expand into a one-dimensional vector. This is the final output.

[0037] Step 302: Extract feature data using a bidirectional gated loop control unit;

[0038] A Bidirectional Gated Recurrent Unit (BiGRU) is employed to perform deep temporal dependency modeling of time-series features, simultaneously capturing forward and backward dynamic changes in market data. This BiGRU is based on a recurrent neural network architecture and introduces a gating mechanism to mitigate the vanishing gradient problem and enhance the ability to learn long-term dependencies. The BiGRU consists of two GRUs, which compute the hidden states along the forward and backward directions of the time series, respectively, and concatenate the hidden states to form a bidirectional context representation. Its mathematical description is as follows: Given an input temporal feature sequence X = [x1, x2, ..., x...], ... T ]∈R T×d Where T represents the time step length and d represents the feature dimension of each time step. For each time step t∈[1,T], the forward and backward hidden state update formulas of BiGRU are as follows: For forward GRU, For backward GRU Where x t ∈R d The input features at time step t, h t (f) h t (b) These represent the hidden states for the forward and backward directions, respectively. The superscript f indicates the forward direction, the superscript b indicates the backward direction, and z represents the hidden state. t (·) R represents the update gate vector at time step t. t ·(·) This indicates that the gate vector is reset at time step t. W represents the candidate hidden state vector at time step t. z (·) ∈R h×d W represents the input data weight matrix used to compute the update gate. r (·) ∈R h×d W represents the input data weight matrix for calculating the reset gate. h (·) ∈R h×d U represents the input data weight matrix used to compute the candidate hidden states; z (·) ∈R h×h U represents the hidden state weight matrix used to compute the update gate. r (·) ∈R h×h U represents the hidden state weight matrix for calculating the reset gate. h (·) ∈R h×h This represents the hidden state weight matrix used to compute candidate hidden states, where h represents the dimension of the hidden state, and d represents the feature dimension at each time step; b z (·) b represents the bias vector for calculating the update gate. r (·) b represents the bias vector for calculating the reset gate. h (·) represents the bias vector used to compute the candidate hidden state; ⊙ represents the Hadamard product (element-wise multiplication); σ(·) represents the sigmoid activation function; tanh(·) represents the Tanh activation function. Finally, the bidirectional GRU output at time step t is represented as... Where [;] denotes vector concatenation, and the output of the entire time series is represented as

[0039] Step 303: Extract key temporal features using a weighted approach based on an attention mechanism;

[0040] To further enhance feature representation capabilities, an attention mechanism is introduced based on the output of the Bidirectional GRU (BiGRU) module. This mechanism weights and converges temporal features, improving the model's ability to focus on key time steps. The attention mechanism adaptively learns the importance weights of each time step for the final task, thereby strengthening useful information and suppressing useless information. Let the output of the BiGRU module be H = [h1, h2, ..., h...]. T ], where h t ∈R 2h The bidirectional context representation of each time step t is used. The computation process of the attention mechanism is as follows: First, the hidden state h of the first time step t is... t Calculate its attention score e t That is, e t =υ Τ tanh(W a h t +b a ); where W a Let b represent the attention weight matrix. a Let ν denote the attention bias vector, ν denote the attention score vector, tanh(·) denote the hyperbolic tangent activation function, and e t Let be the attention score at time step t, representing the importance of that time step; then, the Softmax function is used to normalize the scores of all time steps to obtain the attention weight α. t : Where α t ∈(0,1), and This represents the attention distribution weights for each time step t. Then, a weighted summation method is used to aggregate the hidden states of all time steps into a context representation vector s: Here, s represents the feature representation after fusing attention weights, which enhances the expressive power for key time steps. Finally, the attention context vector s is projected onto the target space through a fully connected layer to generate the final output: Among them W fc Let b represent the weight matrix of the fully connected layer. fc This represents the bias vector of the fully connected layer. This is the final output of the attention mechanism;

[0041] Step 304: Fusion of multi-source temporal features

[0042] To fully leverage the capabilities of Convolutional Neural Networks (CNNs) in extracting local short-term features, Bidirectional Gated Recurrent Units (BiGRUs) in modeling temporal dependencies, and attention mechanisms in selecting key time steps, these three types of features are modeled and integrated in a unified manner to form a more discriminative joint feature representation. Let the pooled feature representation extracted by the CNN module be... The features extracted by the BiGRU module are represented as follows: The context vector processed by the attention mechanism from the BiGRU module is represented as follows: The comprehensive feature vector F is obtained by vector concatenation: Subsequently, to enhance the nonlinear expressive power and the interaction between features, the fused feature vector F is fed into a fully connected layer for projection processing: Among them W f The weight matrix representing the fused features, b f Let σ(·) represent the bias vector, and let σ(·) represent the nonlinear activation function. This represents the final fused feature vector;

[0043] Step 400: Construct a multi-agent trading system;

[0044] Step 401, construct the trading environment;

[0045] To achieve a more realistic reinforcement learning trading simulation system across multiple market conditions, a dynamic and interactive trading environment was constructed based on historical price data. This environment not only simulates the impact of trading behavior on funds and positions but also imposes strict constraints on each transaction to better reflect real-world trading scenarios. Firstly, at the transaction execution level, to prevent unrealistic behavior, the system introduces a position and short position restriction mechanism. Specifically, if the agent does not currently hold a position, it is prohibited from executing a sell operation. Secondly, a buy operation must meet the requirement that the transaction amount is less than or equal to the current available funds, effectively preventing short selling or buying without capital support and ensuring the logical integrity of the simulated trading. Furthermore, to simulate the cost losses in real trading, a transaction fee mechanism is implemented in the environment. The transaction fee calculation formula for each transaction is as follows: Among them, C t This represents the closing price at the current trading time t, D is the transaction share, and β is the set commission rate, which is set to 0.1 by default. This fee model is used both to calculate the total expenditure when buying and to deduct the total revenue when selling, ensuring that the agent weighs the costs and benefits when making trading decisions.

[0046] Step 402, Defining the reward function;

[0047] The reward function is designed by combining market state perception with historical trend review. The reward function has a unified calculation logic across bull, bear, and sideways market conditions, but employs different evaluation methods depending on the historical data window. When the trading environment step count (current_step) > 30, the system incorporates the average closing price of the most recent 30 days. The difference between the current price and the current price is calculated as the reward score, defined as follows: Conversely, when the transaction is in its initial stage (less than 30 steps), the closing price C of the previous trading day is used. t-1 To replace the average trend value: Reward = (C t -C t-1 The reward function is calculated as () × D, where Reward represents the reward score and D represents the trading share. By smoothly switching between different trading periods, the responsiveness of the reward function to price trends can be effectively improved, avoiding the impact of excessive short-term fluctuations or long-term delays on strategy evaluation. Especially in bull markets, this mechanism strengthens the positive returns of trend-following operations; while in bear markets and range-bound markets, it is used in conjunction with a stop-loss mechanism to punish unreasonable trading behavior.

[0048] Step 403, Design of dynamic stop-loss mechanism;

[0049] In highly volatile bear and sideways market scenarios, simple price changes are insufficient to guarantee risk control. Therefore, a dynamic stop-loss mechanism based on the Average True Range (ATR) is introduced to enhance the model's risk perception capability. ATR is a classic volatility indicator calculated from historical prices over the past 30 days to measure current market volatility. If the current price change exceeds a certain multiple of the ATR (denoted as ζ = 1.5), a stop-loss condition is triggered: |C t -C t-1 |>ζ×ATR, where C t The closing price at time step t represents the closing price. Once this condition is met, a dynamic stop-loss mechanism will be immediately executed to adjust the reward and give a certain penalty score to simulate the automatic stop-loss behavior in reality, so that the agent can identify risks and avoid extreme losses more promptly when there are abnormal fluctuations.

[0050] Step 404, Adaptive position control;

[0051] Adaptive position control based on market conditions dynamically adjusts trading size to maximize returns and minimize risks. Specific rules are as follows: In a bull market, all account funds are used to buy in, maximizing participation in the upward trend. Bull market trading volume is calculated as follows: In a bear market, only 1 / 4 of the account funds should be used for trading to reduce risk exposure. The calculation of the trading quota in a bear market is as follows: In a volatile market, use half of your account funds for trading, balancing offense and defense: Where B represents the current account balance, P t The current trading price is used. This mechanism simulates the typical behavior of human traders under different market sentiments, enabling reinforcement learning agents to dynamically adjust risk exposure and improving the stability and practicality of the strategy.

[0052] Step 500: Multiple agents execute transactions;

[0053] Step 501: Design a priority experience pool;

[0054] To improve the learning efficiency of the agent and focus on key experience samples, a priority experience replay mechanism is introduced into the agent's design. Traditional experience replay treats all samples with equal weight, while the priority experience pool uses the temporal difference error (TD error) of each sample as a priority metric, focusing on high-information samples that the model has not yet mastered. In the priority experience pool design, the priority calculation formula for each sample is as follows: p i =(δ i +ε) λ , where p i δ represents the priority of the i-th sample. i Let represent the TD error of the i-th sample, λ∈[0,1] control the priority influence, with a default value of 0.6, and ε is a perturbation term to prevent probability 0. During sampling, the probability of a sample being selected is: In addition, to eliminate sampling bias, an importance sampling weight w will be introduced. i The learning process is revised to be defined as: Where κ∈[0,1] controls the degree of correction for sampling bias, gradually increasing to 1 during training, eventually achieving complete correction. This priority-based empirical mechanism allows the model learning process to focus more on key experiences, while maintaining training stability and generalization ability;

[0055] Step 502, Design a dynamic exploration strategy

[0056] By designing a dynamic exploration and utilization balancing mechanism, the exploration and convergence quality of the strategy can be improved. In the early stages of training, a larger exploration rate ∈ is set to encourage exploration, allowing the agent to quickly learn environmental features. As training progresses, the value of ∈ gradually decays, according to the following decay formula: ∈ t+1 =max(∈ min ,∈ t ·∈ decay ), where ∈ decay Represents the attenuation coefficient, ∈ min The minimum exploration rate is represented to ensure that the agent's performance eventually tends towards a stable strategy. Meanwhile, the κ value of the priority experience pool gradually increases to 1.0 during training to improve the fairness of sample sampling and learning stability. This dynamic two-factor mechanism (ε decay + κ enhancement) achieves a smooth transition from exploration to exploitation to correction to stability, effectively enhancing the agent's adaptability and generalization ability in complex markets.

[0057] Step 503, construct the agent agent;

[0058] In the agent's policy network design, a Dueling DQN architecture is adopted to more finely distinguish between state value and action advantage. The network structure consists of three parts: a feature extractor, a value stream, and an advantage stream. The feature extractor is a multi-dimensional feature extractor constructed in step 300, used to extract deep semantic features from the feature data processed by the sliding window in step 105. The value stream predicts the current value V(s) through a multi-layer neural network, and the advantage stream estimates the relative advantage A(s,a) of each action through a parallel network. Finally, the Q-values ​​are fused as follows: Q(s,a) = V(s) + A(s,a), where Q(s,a) represents the Q-value (action value) of choosing action a in the current state s. The state value function V(s) is used to evaluate the overall good or bad in state s, regardless of the action taken. The advantage function A(s,a) represents the evaluation of the advantage of a specific action a taken in state s over other actions. This structure significantly improves the expressive power of the policy network in the state space, especially performing well in scenarios with multiple similar states but different policies.

[0059] Step 504: Construct a multi-agent transaction model;

[0060] To mitigate trading risks arising from misjudgments by a single strategy, multiple agents for stock trading are constructed based on the Dueling DQN network. First, each agent is trained independently using the training set processed by the sliding window in step 105, based on the multi-dimensional feature extractor constructed in step 300, ensuring the independence of each agent. The training process takes place in the market environment constructed in step 400. Simultaneously, the reward function defined in step 402 is used for evaluation at each time step, continuously improving the system. When the profit or loss reaches the dynamic threshold set in step 403, a dynamic stop-loss mechanism is triggered, resulting in a closed position. Then, the state environment information of each agent in the validation set provided by the multi-dimensional feature extractor in step 105 is used to validate the training effect of each agent. The model parameters of the best-performing agent are selected as the model parameters for that agent. Next, all trained and validated agents are applied to the constructed trading environment, independently perceiving the market state and making decisions at each time step t. Subsequently, the actions of multiple strategies are aggregated through a voting mechanism, and the optimal decision is finally selected for execution. Specifically, assuming there are N agents in total, the i-th agent, in the current state s... t The action selection is a i Then, the voting frequency of all actions will be counted: in This is an indicator function that returns whether the actions are the same. If there are ties, the selection is weighted according to the average Q-value of the individual policies. This mechanism can effectively improve the robustness and generalization ability of the agent's decision-making.

[0061] An adaptive multi-agent stock trading device based on three-state market perception includes: at least one processor and a memory communicatively connected to the at least one processor;

[0062] The memory stores instructions that can be executed by the processor, and these instructions are executed by the at least one processor to cause the at least one processor to perform the aforementioned adaptive multi-agent stock trading method based on three-state market perception.

[0063] Compared with existing methods, the beneficial effects of the present invention are as follows:

[0064] This invention proposes an adaptive multi-agent stock trading method based on three-state market perception, aiming to improve the stability and sustainability of returns in complex and volatile stock market environments. The entire method constructs an intelligent trading framework with strong adaptability, high risk control capabilities, and robust decision-making through multi-stage model design and integrated optimization. The first part, at the data level, standardizes and preprocesses historical trading data based on stock market technical analysis theory, constructing a feature label system including core indicators such as opening price, closing price, highest price, lowest price, and trading volume. Multiple technical indicators are further introduced to enhance feature expression capabilities, providing unified, standardized, and structurally complete input data for subsequent modeling. The second part, by constructing a three-state market perception machine, achieves dynamic identification of market states and significantly enhances environmental perception capabilities. This part employs a volatility-based quantitative analysis method, constructing a market volatility index by calculating the historical closing price change rate and sliding standard deviation. Combined with a preset threshold strategy, the stock market is divided into three typical states: bull market, bear market, and sideways market, thus providing real-time and accurate market environment input for the multi-agent system. The system can dynamically perceive changes in market conditions, constructing a more timely and market structure-understanding trading system environment, effectively improving the flexibility and adaptability of trading strategies. The third part constructs a multi-dimensional feature extraction model integrating convolutional neural networks (CNN), bidirectional gated recurrent units (BiGRU), and attention mechanisms. This model aims to perform deep, multi-faceted feature modeling and expression of financial time series data, enhancing the model's ability to perceive and discriminate changes in market conditions. The fourth part constructs a dynamic and realistic reinforcement learning trading environment, simulating actual trading constraints such as capital limits, transaction fees, and position rules. The environment is designed with an adaptive reward function based on market conditions (bull market, bear market, sideways market), taking into account both short-term fluctuations and long-term trends, improving the strategy's responsiveness. Simultaneously, a dynamic stop-loss mechanism based on ATR is introduced to strengthen risk control. Finally, the environment automatically adjusts position size according to market conditions, simulating typical human investor behavior under different market conditions, enhancing the stability and practicality of the trading strategy. The fifth part designs a multi-agent trading system and its key mechanisms. A priority experience pool is introduced, allowing the agent to focus more on important and difficult-to-master experience samples during training, improving learning efficiency and stability. A dynamic exploration strategy is employed to gradually decay the exploration rate and correct sampling bias during training, ensuring a smooth transition to a stable model. The agent uses a Dueling DQN architecture to estimate state value and action advantage separately, enhancing the expressive power of the policy network. After multiple agents are trained independently, a voting mechanism is used to merge decisions, improving the robustness and generalization ability of the overall trading strategy and effectively reducing the risk of misjudgment by a single model.

[0065] This invention constructs a dynamic and highly realistic trading environment, combined with multi-market state judgments, making the training process closer to the real market and improving the practicality and adaptability of the strategy. Secondly, the design of the reward function and dynamic stop-loss mechanism enhances the model's risk perception and profit capture capabilities under different market conditions, achieving a balance between risk control and profit maximization. Thirdly, adaptive position control and a multi-agent architecture improve the strategy's flexibility and robustness, effectively reducing the risk of misjudgment from a single model. The introduction of a priority experience pool and dynamic exploration strategy significantly improves training efficiency and the strategy's generalization ability, enabling the agent to quickly master key trading experiences and stably converge to a superior strategy. Finally, the adoption of a Dueling DQN structure refines the evaluation of value and advantage, enabling the agent to make more accurate decisions in complex and ever-changing market environments. Attached Figure Description

[0066] Figure 1 This is a flowchart illustrating the implementation of the adaptive multi-agent stock trading method based on three-state market perception provided by the present invention.

[0067] Figure 2 This is the flowchart for implementing step 100;

[0068] Figure 3 This is the flowchart for implementing step 200;

[0069] Figure 4 This is the flowchart for implementing step 300;

[0070] Figure 5 This is the flowchart for implementing step 400;

[0071] Figure 6 This is the flowchart for implementing step 500. Detailed Implementation

[0072] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0073] Figure 1 This is a flowchart illustrating the implementation of the adaptive multi-agent stock trading method based on three-state market perception provided by the present invention. The specific steps are as follows:

[0074] Step 100 involves crawling stock data from the Yahoo platform to obtain trading information for indices, futures, and individual stocks. This data is then parsed, filtered, and cleaned to create a structured trading dataset. Next, based on indicators such as opening price, highest price, lowest price, closing price, and trading volume, technical indicators such as moving averages, rates of change, and volatility are calculated to construct a feature label set. Subsequently, Min-Max standardization is used to unify the data scale, and time-series samples are constructed using a sliding window approach to provide standardized and stable input data for subsequent modeling.

[0075] like Figure 2 As shown, step 100 includes steps 101 to 105:

[0076] Step 101: Data source determination. The required index, futures, and individual stock datasets are obtained from Yahoo.

[0077] Step 102, the web crawler, through the parsing function S d =Parse(html), extract function E d =Extract(S d This is used to implement web scraping; where html is the original webpage, and S... d E represents the parsed data obtained after the parsing function Parse(·). d This represents the parsed data obtained after the analytic function Extract(·).

[0078] Step 103: Filter and clean stock trading data. Use text processing techniques to clean and normalize the data. d =Clean(Fliter(E) d Ensure data consistency and accuracy, and address data quality issues.

[0079] Step 104: Construct feature labels. Based on information such as opening price, highest price, closing price, closing price, and trading volume of stock data, calculate moving average, exponential moving average, closing price change rate, and 30-day closing price volatility to construct feature labels S = {O, H, L, C, V, M} for model training. 30 M 60 E 90 ,P r ,Ψ 30}

[0080] Step 105: Data normalization and windowing. The original feature data is linearly transformed using Min-Max standardization, calculated as follows: Mapping feature values ​​to the closed interval [0,1] effectively eliminates the dimensional differences between different features, improves numerical stability, and prevents larger numerical features from dominating the model training process, thus ensuring a balanced contribution from each dimension. Simultaneously, a sliding window technique is used to construct time series samples. Each time window S... t The feature data, S, includes the current time t and the previous 30 time steps. t =[s t-30 ,s t-29 ,…,s t-1 This provides a standardized input format for subsequent model training.

[0081] Step 200: Construct a three-state market perception machine.

[0082] like Figure 3 As shown, the volatility quantification analysis method is used to calculate price volatility and combine it with threshold judgment rules to divide the market into three market states: bull market, bear market and sideways market, so as to achieve adaptive perception of the market environment.

[0083] Step 201: Perform a difference operation on the closing price sequence to calculate the daily closing price change rate, thus measuring short-term market price volatility. Given a closing price sequence {C1, C2, ..., C...} t} Calculate the rate of change of closing price at each time step.

[0084] Step 202: Based on the closing price change rate sequence, calculate volatility using a sliding window standard deviation to quantify the severity of price fluctuations. Given a window length N = 30 (i.e., 30 trading days), calculate the volatility on day t.

[0085] Step 203: Classify market states based on thresholds. Using a preset volatility threshold, classify market state S... t Dynamically divided into three categories: Where 0 represents a bull market, 1 represents a bear market, and 2 represents a sideways market; θ bull =2% is the bull market threshold, indicating a low-volatility market environment, usually accompanied by steady growth, θ bear =5% is the threshold for a bear market, indicating a highly volatile market environment, usually accompanied by sharp declines. Volatility in between is classified as a sideways market, reflecting a lack of clear market trend.

[0086] Step 204: Output market state signals. Through quantitative analysis of market fluctuation characteristics, the daily market state label {State1,State2,…,S} is finally output. tatet}, where each State t ∈{0,1,2} represents the market status classification result on the t-th trading day.

[0087] Step 300: Construct a multi-dimensional feature extractor.

[0088] like Figure 4 As shown, a multi-dimensional feature extractor is constructed by using a convolutional neural network (CNN), a bidirectional gated recurrent control unit (BiGRU), and an attention mechanism to deeply mine and analyze financial time series data.

[0089] Step 301: Use a convolutional neural network (CNN) to perform deep feature extraction on financial time-series features to capture local patterns and short-term dependencies in market data. A multi-layer convolutional architecture is employed, using a hierarchical feature transformation mechanism to map from raw data to high-order abstract features.

[0090] Step 302 involves using a bidirectional gated recurrent unit (BiGRU) to perform deep temporal dependency modeling on the temporal features, thereby simultaneously capturing forward and backward dynamic changes in the market data. Based on the recurrent neural network architecture, a gating mechanism is introduced to alleviate the gradient vanishing problem and enhance the ability to learn long-term dependencies.

[0091] Step 303 introduces an attention mechanism based on the output of the bidirectional GRU (BiGRU) to weight and converge temporal features, thereby enhancing the model's ability to focus on key time steps. The attention mechanism can adaptively learn the importance weights of each time step to the final task, thereby strengthening useful information and suppressing useless information.

[0092] Step 304: To fully utilize the Convolutional Neural Network (CNN)'s ability to extract local short-term features, the Bidirectional Gated Recurrent Unit (BiGRU)'s ability to model temporal dependencies, and the attention mechanism's ability to select key time steps, the above three types of features are uniformly modeled and integrated to form a more discriminative joint feature representation. The comprehensive feature vector F is obtained through vector concatenation: Subsequently, to enhance the nonlinear expressive power and the interaction between features, the fused feature vector F is fed into a fully connected layer for projection processing:

[0093] Step 400: Construct a multi-agent trading system.

[0094] like Figure 5As shown, a reinforcement learning trading environment closely resembling the real market was constructed, encompassing key designs such as trading rules, reward mechanisms, stop-loss strategies, and position control. First, the environment simulates real capital flows, incorporating position limits and transaction fee mechanisms to ensure compliance and cost considerations in trading. Second, the reward function is dynamically adjusted based on market conditions, combining short-term price trends with long-term averages to enhance the strategy's sensitivity to price changes. To strengthen risk management, a dynamic stop-loss mechanism based on ATR is introduced to promptly limit losses in highly volatile market conditions. Finally, the trading position ratio is dynamically adjusted according to bull market, bear market, and sideways market scenarios, simulating the risk preferences of human investors to achieve a balance between return and risk, effectively improving the strategy's practicality and stability.

[0095] Step 401: Constructing the trading environment. A dynamic and interactive trading environment was built based on historical price data. This environment not only simulates the impact of trading behavior on funds and positions but also imposes strict constraints on each transaction to better reflect real trading scenarios. First, at the transaction execution level, to prevent unrealistic behavior, the system introduces a position and short position restriction mechanism. If the agent does not currently hold a position, it is prohibited from executing a sell operation. Second, the current buy operation must meet the requirement that the transaction amount is less than or equal to the current available funds, which can effectively avoid short selling or buying without capital support, ensuring the logical integrity of the simulated trading. In addition, to simulate the cost losses in real trading, a transaction fee mechanism is set up in the environment. The transaction fee calculation formula for each transaction is as follows:

[0096] Step 402, the reward function is designed by combining market state perception with historical trend review. The reward function has a unified calculation logic across bull, bear, and sideways market conditions, but different evaluation methods are used depending on the historical data window. When the trading environment step count `current_step` > 30, the system incorporates the average closing price of the most recent 30 days. The difference between the current price and the current price is calculated as the reward score, defined as follows: Conversely, when the transaction is in its early stages (less than 30 steps), the closing price C of the previous trading day is used. t-1 To replace the average trend value: Reward = (C t -C t-1 The )×D mechanism, by smoothly switching between different trading cycles, can effectively improve the responsiveness of the reward function to price trends, avoiding situations where excessive short-term fluctuations or long-term delays affect strategy evaluation. Especially in bull markets, this mechanism strengthens the positive returns of trend-following operations; while in bear markets and range-bound markets, it is used in conjunction with a stop-loss mechanism to punish unreasonable trading behavior.

[0097] Step 403, the design of the dynamic stop-loss mechanism, introduces a dynamic stop-loss mechanism based on the Mean True Range (ATR) to enhance the model's risk perception capability. If the current price change exceeds a certain multiple of the ATR (let's say ζ = 1.5), it is considered a trigger condition for the stop-loss: |C t -C t-1 |>ζ×ATR, once this condition is met, a dynamic stop-loss mechanism will be immediately executed, adjusting the reward and giving a certain penalty score to simulate real-world automatic stop-loss behavior, enabling the agent to identify risks and avoid extreme losses more promptly when fluctuations are abnormal.

[0098] Step 404, Adaptive Position Control: Adaptive position control is implemented based on market conditions to dynamically adjust the trading size, thereby maximizing returns and minimizing risks. In a bull market environment, all account funds are used to buy in, maximizing participation in the upward trend. The bull market trading share is calculated as follows: In a bear market, only 1 / 4 of the account funds should be used for trading to reduce risk exposure. The calculation of the trading quota in a bear market is as follows: In a volatile market, use half of your account funds for trading, balancing offense and defense: This enables reinforcement learning agents to dynamically adjust their risk exposure, improving the stability and practicality of the strategy.

[0099] Step 500 constructs a multi-agent reinforcement learning trading system to improve the stability and generalization ability of the strategy. It focuses on key samples through a priority experience pool, achieves a smooth transition from exploration to convergence through a dynamic exploration mechanism, enhances the expressiveness of the strategy through a Dueling DQN structure, and finally adopts multi-agent voting decision-making, integrating multiple strategies to reduce the risk of misjudgment by a single strategy and effectively improve overall trading performance.

[0100] Step 501: Design a priority experience pool. The time difference error (TD error) of each sample is calculated as a priority metric to focus on high-information samples that the model has not yet mastered. The priority calculation formula for each sample in the priority experience pool design is as follows: p i =(δ i +ε) λ To eliminate sampling bias, an importance sampling weight w will be introduced. i The learning process is revised to be defined as: Here, κ∈[0,1] controls the degree of correction for sampling bias, which gradually increases to 1 during the training process, eventually achieving complete correction. This priority-based empirical mechanism allows the model learning process to focus more on key experiences, while maintaining training stability and generalization ability.

[0101] Step 502: Design a dynamic exploration strategy. By designing a dynamic exploration and utilization balance mechanism, the exploration and convergence quality of the strategy can be improved. In the early stages of training, a larger exploration rate ∈ is set to encourage exploration, so that the agent can quickly learn environmental feature information. As training progresses, the value of ∈ will gradually decay, and the decay formula is as follows: ∈ t+1 =max(∈ min ,∈ t ·∈ decay ), where ∈ decay The default value is 0.995, ∈ min The β value is set to 0.01 to ensure that the agent's performance eventually stabilizes with the strategy. Meanwhile, the β value of the priority experience pool gradually increases to 1.0 during training to improve the fairness of sample sampling and learning stability. This dynamic two-factor mechanism (ε decay + κ enhancement) achieves a smooth transition from exploration to exploitation to correction to stabilization, effectively enhancing the agent's adaptability and generalization ability in complex markets.

[0102] Step 503: Construct an intelligent agent using a Dueling DQN architecture to more finely distinguish state value and action advantage. The network structure consists of three parts: a feature extractor, a value stream, and an advantage stream. The feature extractor extracts deep semantic features from the original market state. The value stream predicts the current value V(s) through a multi-layer neural network. The advantage stream estimates the relative advantage A(s,a) of each action through a parallel network. Finally, the Q-values ​​are fused as follows: Q(s,a) = V(s) + A(s,a).

[0103] Step 504: Construct a multi-agent trading model. Based on the Dueling DQN network, construct multiple agents for stock trading. First, train each agent independently using the environmental state information from the training set provided by the multi-dimensional feature extractor to ensure independence. Then, validate the training performance of each agent using the environmental state information from the validation set provided by the multi-dimensional feature extractor. Select the model parameters with the best performance for each agent as its model parameters. Next, apply all trained and validated agents to the constructed trading environment. At each time step t, each agent independently perceives the market state and makes a decision. Subsequently, aggregate the actions of multiple strategies through a voting mechanism, and finally select the optimal decision to execute.

[0104] Based on the solution of this invention, experimental analysis was conducted on different datasets, as detailed below:

[0105] Table 1: Experimental results on the exponential dataset

[0106]

[0107]

[0108] Table 2: Experimental results on the futures dataset

[0109]

[0110]

[0111] Table 3: Experimental results on individual stock datasets

[0112]

[0113]

[0114] Table 1 presents a comprehensive comparative analysis of the experimental results of the method of this invention on index stock market datasets (S&P 500, SSE Composite, SZI, N100). Starting from four indicators—cumulative return, annualized return, maximum drawdown, and Sharpe ratio—it clarifies the performance improvement achieved by the method of this invention compared to other traditional machine learning models (decision trees, support vector machines) and reinforcement learning models (Q-learning, GRU-DQN, CNN-DQN), clearly demonstrating the advantages of this invention in various market environments.

[0115] On the S&P 500 dataset, the method of this invention achieved a cumulative return of 49.07%, significantly higher than all comparable models. Specifically, it improved by 30.69% (a 167% improvement) compared to the buy-and-hold strategy, by 25.92% compared to the traditional best support vector machine model, and by 39.28% compared to the best deep learning model (GRU-DQN). In terms of annualized return, this invention achieved 14.23%, significantly outperforming all comparable models, with improvements ranging from 7.04% (compared to SVM) to 13.74% (compared to Q-learning). Regarding the Sharpe ratio, this invention achieved 1.57, an improvement of 0.97 compared to the second-best model (GRU-DQN) and more than double that of the traditional model, indicating its superior performance in balancing return and risk. The maximum drawdown was 9.87%, slightly higher than SVM (7.49%) and GRU-DQN (8.18%), but still significantly better than Buy & Hold (19.78%) and Q-learning (9.77%), demonstrating more robust overall performance. In the SSE Composite dataset, where the market trended downwards and several models experienced negative returns, this invention still achieved a cumulative return of 12.78%, far exceeding all compared models, with a maximum improvement of 27.82% (compared to Buy & Hold) and an improvement of 11.99% compared to the positive-returning Q-learning. The annualized return was 4.09%, a 5.71% improvement compared to the best traditional model (decision tree) and a 5.98% improvement compared to the deep model GRU-DQN. Regarding the Sharpe ratio, this invention achieved 0.84, the only model to achieve a positive value, significantly outperforming all other compared methods (with a maximum improvement of 2.43 and a minimum of 1.05), demonstrating its advantages in risk control and stable profitability. The maximum drawdown was 13.62%, 16.62% lower than the buy-and-hold model, but slightly higher than decision trees and CNN-DQN, maintaining a good level of robustness. For the SZI dataset, market volatility was extreme, and the performance of each model varied significantly. Our method achieved a cumulative return of 18.88%, outperforming all comparable models, especially compared to GRU-DQN (which only achieved 3.85%), where it improved by 15.03%. Compared to other models exhibiting negative returns, the highest improvement reached 50% (compared to buy-and-hold). In terms of annualized return, our method achieved 5.94%, 4.67% higher than GRU-DQN and 9.15% higher than SVM. The Sharpe ratio reached 0.52, 0.54 higher than the best-performing deep model (GRU-DQN at -0.02), while improvements to other models all exceeded 1.0, indicating that our method can maintain a positive risk-reward ratio under extreme market volatility. Although the maximum drawdown is slightly higher than CNN-DQN (4.56% higher), it is still significantly lower than models such as Q-learning and buy-and-hold.On the N100 dataset, this invention achieved a cumulative return of 35.92%, the highest among all models, outperforming the best-performing deep learning model CNN-DQN (21.63%) by 14.29%, and exceeding traditional models such as SVM and decision trees by over 40%. The annualized return reached 10.77%, 4.03% higher than CNN-DQN and 5.25% higher than GRU-DQN. The Sharpe ratio was 1.39, second only to CNN-DQN's 1.40, but its advantage lay in superior maximum drawdown control at only 6.75%, slightly higher than CNN-DQN's 3.48%, but far superior to Q-learning (25.13%) and buy-and-hold (17.99%). Overall, this invention achieved a good balance between return stability and drawdown control.

[0116] As shown in Table 2, on the futures dataset, the method of this invention still outperforms the comparative models in terms of profitability, risk control, and risk-reward ratio. It achieves a cumulative return of 64.82% on the Crude dataset, the highest among all models. Compared to the best-performing deep learning model GRU-DQN (36.62%), it improves by 28.2%; compared to the traditional SVM model, it improves by 52.24%; and compared to the baseline model "Buy and Hold," it improves by 74.81%, significantly outperforming the comparison. The annualized return also reaches 18.12%, an improvement of 7.16% compared to GRU-DQN (10.96%), representing an improvement of approximately 65%. Regarding maximum drawdown, this invention achieves 24.80%, slightly higher than GRU-DQN's 13.31%, but 19.54% lower than "Buy and Hold," effectively controlling drawdown while maintaining returns. The Sharpe ratio is 0.60, slightly lower than GRU-DQN's 1.36, but significantly better than other models, especially showing a substantial improvement over Q-learning (-2.93), indicating more stable returns. The Coffee dataset represents a highly volatile market, where traditional models generally suffer significant losses. Our method achieved a cumulative return of 33.61% on this dataset, far exceeding all other models, with CNN-DQN achieving only 0.55% and all others showing negative returns. This represents a 75.98% improvement over SVM (-42.37%) and a 51.87% improvement over GRU-DQN (-18.26%). The annualized return is 10.14%, a 9.96% improvement over CNN-DQN, and improvements between 16% and 27% compared to Q-learning (-6.46%) and traditional models. The Sharpe ratio is 3.33, the highest among all models and the only one to achieve a positive value significantly higher than 1, significantly outperforming GRU-DQN (-1.66) and CNN-DQN (-0.75), indicating that this invention can still achieve a strong risk-reward ratio in highly volatile markets. The maximum drawdown is 23.35%, slightly higher than CNN-DQN (18.80%) and GRU-DQN (28.13%), but significantly lower than traditional models (nearly 47%), demonstrating that this invention effectively overcomes the bottleneck of traditional models' inability to generate profits while maintaining controllable drawdown risk. On the Gold dataset, it achieved a cumulative return of 17.57%, outperforming all other models. The second best performer was "Buy and Hold" with only 1.81%, while CNN-DQN was even more negative (-11.15%), resulting in an improvement of 15.76%-31.93%. The annualized return is 5.54%, which is 4.94% higher than the traditional optimal model (buy and hold) and 10.29% higher than the optimal deep model GRU-DQN.The Sharpe ratio reached 1.22, the only positive model, significantly outperforming all comparison models (all below 0), especially SVM at -4.81, demonstrating excellent risk control and return stability. The maximum drawdown was 7.82%, the lowest among all models, 5.88% lower than the "buy and hold" approach, and also lower than GRU-DQN (15.49%) and CNN-DQN (15.45%), showcasing the strong risk resistance of this invention in the precious metals market. In the Corn dataset, the method of this invention also achieved the highest cumulative return of 47.17%, leading GRU-DQN (29.59%) and Q-learning (18.80%), with improvements of 17.58% and 28.37% respectively. Compared to SVM (10.78%), it improved by 36.39%, and the improvements of the traditional model and CNN-DQN (9.28%) were also above 35%. The annualized return is 13.75%, which is 4.72% higher than the closest GRU-DQN and 10.75% higher than CNN-DQN. The Sharpe ratio is 1.93, close to GRU-DQN's 1.48, but significantly higher than the traditional optimal models SVM (2.15) and Q-learning (1.05), showing that this invention combines high returns and robustness. The maximum drawdown is 19.40%, comparable to "buy and hold" (19.16%) and CNN-DQN (19.35%), and slightly higher than GRU-DQN (5.38%), but considering its significantly higher returns, the drawdown is still within an acceptable range.

[0117] As shown in Table 3, on the individual stock dataset, the AMZN stock achieved a cumulative return of 228.50%, far exceeding GRU-DQN (145.03%) and SVM (96.10%), with improvements of 83.47% and 132.4% respectively, and a 124.22% improvement over the "buy and hold" strategy. The annualized return was 48.65%, significantly higher than GRU-DQN's 34.81% (an improvement of 13.84%), making it the best among all models. The Sharpe ratio reached 3.54, far higher than all other models, exceeding GRU-DQN's 2.56 by 0.98, indicating that this method not only achieves high returns but also exhibits lower volatility and greater stability. The maximum drawdown was 21.71%, slightly higher than CNN-DQN (9.64%) and SVM (10.13%), but considering its high-return characteristics, this drawdown is within an acceptable range, indicating that this method achieves a good balance between high returns and risk control. On the INTC dataset, this invention achieved a cumulative return of 88.86%, outperforming all comparable models. Specifically, GRU-DQN achieved 46.36%, and CNN-DQN achieved 33.60%, representing significant improvements of 42.5% and 55.26% respectively. The annualized return reached 23.61%, far exceeding GRU-DQN's 13.54% and the "buy and hold" strategy's 12.25%, representing an annualized growth improvement of approximately +10%. The Sharpe ratio is 1.94, higher than all models except SVM (1.92) and CNN-DQN (2.34), and slightly lower than CNN-DQN. However, considering the significant lead in returns, its Sharpe ratio still demonstrates excellent risk-adjusted returns. The maximum drawdown is controlled at 14.01%, lower than "Buy and Hold" (25.68%) and Q-learning (20.22%), and also better than most traditional methods, demonstrating good drawdown control. On the MSFT dataset, the proposed method achieves a cumulative return of 111.05%, higher than CNN-DQN (99.30%) and the "Buy and Hold" strategy (93.01%), representing improvements of 11.75% and 18.04%, respectively. The annualized return is 28.27%, an improvement of 2.42% over CNN-DQN's 25.85% and 3.76% over "Buy and Hold". The Sharpe ratio reached 2.30, the highest among all models in this dataset, an improvement of 1.34 compared to CNN-DQN (0.96), indicating that it achieved lower volatility and better stability while maintaining high returns. The maximum drawdown was 10.52%, one of the lowest among all models, only higher than decision trees (7.80%), but better than GRU-DQN (12.60%) and CNN-DQN (15.73%), demonstrating excellent overall performance. NVDA is a typical high-volatility, high-growth stock.This invention achieved a cumulative return of 447.16%, the highest among all models, significantly exceeding CNN-DQN (257.94%) and GRU-DQN (178.89%), representing improvements of 189.22% and +268.27% respectively. Compared to the "buy and hold" strategy, it improved by 272.09%, demonstrating outstanding performance. The annualized return reached 76.21%, significantly higher than CNN-DQN's 52.97% (23.24%) and GRU-DQN's 40.76% (35.45%). While the Sharpe ratio of 1.65 is lower than SVM (1.83), it still performs superiorly among all deep learning models, combining stability and high returns. The maximum drawdown was 24.58%, slightly higher than CNN-DQN (12.62%) and SVM (14.21%), but considering the significant return growth, the drawdown was within a reasonable range, reflecting that in a high-growth market, the model can withstand greater drawdowns to obtain higher returns.

Claims

1. An adaptive multi-agent stock trading method based on three-state market perception, characterized in that, The steps are as follows: Step 100: Data crawling and preprocessing; Step 200: Construct a three-state market sensing mechanism; Step 300: Construct a multi-dimensional feature extractor; Step 400: Construct a multi-agent trading system; Step 500: Multi-agent execution of transactions.

2. The adaptive multi-agent stock trading method based on three-state market perception according to claim 1, characterized in that, Step 100 is as follows: Step 101: Determine the data source and the area and theme of stock trading; Among them, the data source is determined by selecting Yahoo resources for the index, futures, and individual stock datasets to be obtained; Step 102: Use web scraping technology to obtain stock trading data from the data source; Web crawlers, through parsing function S d =Parse(html), extract function E d =Extract(S d This is used to implement web scraping; where html is the original webpage, and S... d E represents the parsed data obtained after the parsing function Parse(·). d This represents the parsed data obtained after the analytic function Extract(·); Step 103: Filter and clean stock trading data to remove irrelevant information and make the data cleaner and more standardized. Data cleaning and normalization are performed using text processing techniques, denoted as E. d Given the original dataset, the data is filtered using the Filter(E) operation. d By applying filtering rules or conditions, data that meets the requirements or standards is selected from the original data to generate an intermediate dataset F. d Next, the intermediate dataset F d Perform data cleaning operations Clean(F) d This process removes data with missing or incorrect values ​​to ensure data consistency and accuracy, ultimately generating the cleaned dataset C. d The formula is expressed as: C d =Clean(Filter(E d ); where Filter(·) represents the data filtering operation, and Clean(·) represents the data cleaning operation; Step 104, construct feature labels; Based on stock data including opening price, highest price, closing price, lowest price, and trading volume, a feature label S = {O, H, L, C, V, M} is constructed for model training by calculating a moving average, exponential moving average, closing price change rate, and 30-day closing price volatility for the closing price. 30 M 60 E 90 ,P r ,Ψ 30 } where O is the opening price, H is the highest price, L is the lowest price, C is the closing price, V is the trading volume, and M is the lowest price. 30 It is a 30-day simple moving average, M 60 It is a 60-day simple moving average, E 90 It is the 90-day exponential moving average, P r It is the closing price change rate, Ψ 30 It is the 30-day closing price volatility; Use M t This represents the simple moving average over time t; N represents the time period of the moving average, indicating how many time units of price data are used to calculate the average; C t-1 This represents the closing price in the (t-1)th time unit; the simple moving average is calculated using the following formula: E t α represents the exponential moving average over time t; α represents the smoothing factor, representing the weight of the current price and the exponential moving average over the previous time unit; C t E represents the closing price in the t-th time unit. t-1 This represents the exponential moving average over a previous time unit; the formula is: E t =α·C t +(1-α)·E t-1 , P r To represent the rate of change in closing price, use C. t Represents the closing price in the t-th time unit, using C. t-1 Let represent the closing price in the (t-1)th time unit. The formula for calculating the rate of change of the closing price is: Use Ψ t Let μ represent the closing price volatility at time t, and let μ represent the mean of the closing price change rate within a sliding window of size n. The calculation formula is: Let represent the rate of change of the closing price at time step ti; Step 105: Data standardization and windowing processing; Stock feature data is standardized and windowed to optimize data analyzability and time series modeling performance. First, Min-Max standardization is used to linearly transform the feature labels S constructed in step 104. Simultaneously, to capture the short-term dependencies and autocorrelation of stock data, a sliding window technique is used to construct time series samples. Each time window S... t S contains the feature data of the current time t and the previous 30 time steps. t ′=[s t-30 ,s t-29 ,…,s t-1 ], where s t-k Let tk represent the eigenvector of day tk.

3. The adaptive multi-agent stock trading method based on three-state market perception according to claim 1, characterized in that, Step 200 is as follows: Three market perception machines were constructed using volatility quantification analysis to dynamically identify the market status of the stock market and provide environmental perception capabilities for the multi-agent trading system. By calculating price volatility and combining it with threshold judgment rules, the market was divided into three market states: bull market, bear market, and sideways market, so as to achieve adaptive perception capability of the market environment. Step 201: Calculate the closing price change rate; First, a difference operation is performed on the closing price sequence to calculate the rate of change of the daily closing prices, in order to measure short-term price fluctuations in the market; given a closing price sequence {C1, C2, ..., C...} t }, where t is the time step, C t This represents the closing price on day t, and the rate of change of the closing price ΔC. t The calculation formula is as follows: Where ΔC t This represents the percentage change in the closing price on day t compared to day t-1. Data from the first day (t=1) is excluded after calculation because there are no preceding values, ensuring the continuity of the data. Step 202: Calculate the closing price slippage volatility; Based on the closing price change rate series, volatility is calculated using a sliding window standard deviation to quantify the severity of price fluctuations. Given a window length N, the volatility on day t is... in σ represents the mean rate of price change during the window period. t It reflects the market's risk level over the past 30 trading days; due to insufficient data for the previous N-1 time points, missing values ​​need to be removed after calculation. Step 203: Classify market status based on thresholds; By using a preset volatility threshold, the market state S is... t Dynamically divided into three categories: Where 0 represents a bull market, 1 represents a bear market, and 2 represents a sideways market; θ bull The bull market threshold represents a low-volatility market environment, typically accompanied by steady upward movement. θ bear The threshold for a bear market indicates a highly volatile market environment, typically accompanied by sharp declines. Volatility levels between these thresholds are classified as range-bound markets, reflecting a lack of clear market trend. Step 204: Output market status signal; The three-state market perception machine quantitatively analyzes market fluctuation characteristics and ultimately outputs daily market state labels {State1, State2, ..., State}. t }, where each State t ∈{0,1,2} represents the market status classification result on the t-th trading day.

4. The adaptive multi-agent stock trading method based on three-state market perception according to claim 1, characterized in that, Step 300 is as follows: A multi-dimensional feature extractor is constructed using a convolutional neural network (CNN), a bidirectional gated recurrent control unit (BiGRU), and an attention mechanism for in-depth mining and analysis of financial time series. Step 301: Use a convolutional neural network to extract feature data; A convolutional neural network (CNN) is used to perform deep feature extraction on financial time-series features to capture local patterns and short-term dependencies in market data. This CNN employs a multi-layer convolutional architecture, using a hierarchical feature transformation mechanism to map from raw data to higher-order abstract features. Its mathematical expression is as follows: Given the time-series feature label S∈R constructed in step 105... T×d Where T is the time step, d is the feature dimension, R is the set of real stock data, and the convolution kernel W∈R k×m Where k is the convolution width in the time dimension and m is the convolution width in the feature dimension, the convolution operation at time step i can be expressed as: Where j is the index of the convolution kernel in the time dimension, ranging from [1, k], k is the size of the convolution kernel in the time step dimension, i.e., the height of the convolution, l is the index of the convolution kernel in the feature dimension, ranging from [1, m], m is the size of the convolution kernel in the feature dimension, i.e., the width of the convolution kernel, and W j,l X represents the weights of the convolution kernel, indicating the weights of the convolution kernel at position (j,l); i+j-1,l This represents the value of the input data at position (i+j-1,l), where i is the time step index of the current convolution operation; b is the bias, added to each convolution output; Y i It is the output of the convolution operation at time step i, activated by the ReLU activation function ReLU(Y). i ) = max(0,Y i Introducing nonlinearity to generate feature map Z i Subsequently Max pooling is used to downsample the features, with a pooling window size of p, and the output is: P i =max i≤t≤i+p-1 z t Z t Let P represent the feature map at time step t. i This is the output of the pooling result at time step i; finally, the pooled features are flattened and projected onto the target space through a fully connected layer: Among them W fc It is the weight matrix of the fully connected layer, b fc It is the bias vector of the fully connected layer. P is expanded using the flatten(·) function. i Expand into a one-dimensional vector. This is the final output; Step 302: Extract feature data using a bidirectional gated loop control unit; A bidirectional gated recurrent control unit is employed to perform deep temporal dependency modeling of time-series features, simultaneously capturing forward and backward dynamic changes in market data. This bidirectional gated recurrent control unit is based on a recurrent neural network architecture, introducing a gating mechanism to alleviate the gradient vanishing problem and enhance the learning ability for long-term dependencies. The BiGRU consists of two GRUs, which compute the hidden states along the forward and backward directions of the time series, respectively, and concatenate the hidden states to form a bidirectional context representation. Its mathematical description is as follows: Given an input temporal feature sequence X = [x1, x2, ..., x...], ... T ]∈R T×d , where T represents the time step length and d represents the feature dimension of each time step; For each time step t∈[1,T], the forward and backward hidden state update formulas of BiGRU are as follows: For forward GRU, For backward GRU Where x t ∈R d The input features at time step t, h t (f) h t (b) These represent the hidden states for the forward and backward directions, respectively. The superscript f indicates the forward direction, the superscript b indicates the backward direction, and z represents the hidden state. t (·) R represents the update gate vector at time step t. t ·(·) This indicates that the gate vector is reset at time step t. W represents the candidate hidden state vector at time step t. z (·) ∈R h×d W represents the input data weight matrix used to compute the update gate. r (·) ∈R h×d W represents the input data weight matrix for calculating the reset gate. h (·) ∈R h×d U represents the input data weight matrix used to compute the candidate hidden states; z (·) ∈R h×h U represents the hidden state weight matrix used to compute the update gate. r (·) ∈R h×h U represents the hidden state weight matrix for calculating the reset gate. h (·) ∈R h×h This represents the hidden state weight matrix used to compute candidate hidden states, where h represents the dimension of the hidden state, and d represents the feature dimension at each time step; b z (·) b represents the bias vector for calculating the update gate. r (·) b represents the bias vector for calculating the reset gate. h (·) denoted by ; ⊙ represents the bias vector used to compute the candidate hidden state; σ(·) represents the Hadamard product; σ(·) represents the sigmoid activation function; tanh(·) represents the Tanh activation function; finally, the bidirectional GRU output at time step t is represented as h. t =[h t (f) h t (b) ]∈R 2h Where [;] denotes vector concatenation, and the output of the entire time series is represented as Step 303: Extract key temporal features using a weighted approach based on an attention mechanism; An attention mechanism is introduced based on the bidirectional GRU output to perform weighted convergence of temporal features; let the output of the BiGRU module be H = [h1, h2, ..., h T ], where h t ∈R 2h The bidirectional context representation of each time step t is used. The computation process of the attention mechanism is as follows: First, the hidden state h of the first time step t is... t Calculate its attention score e t That is, e t =v Τ tanh(W a h t +b a ); where W a Let b represent the attention weight matrix. a Let ν denote the attention bias vector, ν denote the attention score vector, tanh(·) denote the hyperbolic tangent activation function, and e t Let be the attention score at time step t, representing the importance of that time step; then, the Softmax function is used to normalize the scores of all time steps to obtain the attention weight α. t : Where α t ∈(0,1), and This represents the attention distribution weights at time step t; subsequently, a weighted summation method is used to aggregate the hidden states of all time steps into a context representation vector s: Here, s represents the feature representation after fusing attention weights, which enhances the expressive power for key time steps; finally, the attention context vector s is projected onto the target space through a fully connected layer to generate the final output: Among them W fc Let b represent the weight matrix of the fully connected layer. fc This represents the bias vector of the fully connected layer. This is the final output of the attention mechanism; Step 304: Fusion of multi-source temporal features Let the pooled features extracted by the CNN module be represented as follows: The features extracted by the BiGRU module are represented as follows: The context vector processed by the attention mechanism from the BiGRU module is represented as follows: The comprehensive feature vector F is obtained by vector concatenation: Subsequently, to enhance the nonlinear expressive power and the interaction between features, the fused feature vector F is fed into a fully connected layer for projection processing: Among them W f The weight matrix representing the fused features, b f Let σ(·) represent the bias vector, and let σ(·) represent the nonlinear activation function. This represents the final fused feature vector.

5. The adaptive multi-agent stock trading method based on three-state market perception according to claim 1, characterized in that, Step 400 is as follows: Step 401, construct the trading environment; First, at the transaction execution level, to prevent unrealistic behavior, the system introduces a position and short position restriction mechanism; specifically, if the agent does not currently hold a position, they are prohibited from executing a sell operation. Second, current buy operations must meet the requirement that the transaction amount is less than or equal to the current available funds, which effectively avoids short selling or buying without capital support, ensuring the logical integrity of the simulated trading. Furthermore, to simulate the cost losses in real trading, a transaction fee mechanism is set up in the environment; the transaction fee calculation formula for each transaction is as follows: Among them, C t This represents the closing price at the current trading time t, D is the share of the transaction, and β is the set commission rate. This fee model is used both to calculate the total expenditure when buying and to deduct the total revenue when selling, ensuring that the agent weighs the costs and benefits when making trading decisions. Step 402, Defining the reward function; When the number of steps in the trading environment (current_step) exceeds 30, the system introduces the average closing price of the most recent 30 days. The difference between the current price and the current price is calculated as the reward score, defined as follows: Conversely, when the transaction is in its initial stage (less than 30 steps), the closing price C of the previous trading day is used. t-1 To replace the average trend value: Reward = (C t -C t-1 )×D, where Reward represents the reward score and D represents the transaction share; Step 403, Design of dynamic stop-loss mechanism; A dynamic stop-loss mechanism based on Mean True Range (ATR) is introduced to enhance the model's risk perception capability. ATR is calculated from historical prices over the past 30 days and is used to measure current market volatility. If the current price change exceeds a certain multiple of the ATR, it is considered a trigger for the stop-loss condition: |C t -C t-1 |>ζ×ATR, where C t The closing price at time step t represents the closing price. Once this condition is met, a dynamic stop-loss mechanism will be immediately executed to adjust the reward and give a certain penalty score to simulate the automatic stop-loss behavior in reality, so that the agent can identify risks and avoid extreme losses more promptly when there are abnormal fluctuations. Step 404, Adaptive position control; In a bull market environment, all account funds are used to buy in, maximizing participation in the upward trend. The bull market trading share is calculated as follows: In a bear market, only 1 / 4 of the account funds should be used for trading to reduce risk exposure. The calculation of the trading quota in a bear market is as follows: In a volatile market, use half of your account funds for trading, balancing offense and defense: Where B represents the current account balance, P t This is the current trading price.

6. The adaptive multi-agent stock trading method based on three-state market perception according to claim 1, characterized in that, The specific steps of step 500 are as follows: Step 501: Design a priority experience pool; In the priority experience pool design, the priority calculation formula for each sample is as follows: p i =(δ i +ε) λ , where p i δ represents the priority of the i-th sample. i Let represent the TD error of the i-th sample, λ∈[0,1] control the priority influence, and ε is a perturbation term to prevent a probability of 0; during sampling, the probability of a sample being selected is: In addition, to eliminate sampling bias, an importance sampling weight w will be introduced. i The learning process is revised to be defined as follows: Where κ∈[0,1] controls the degree of correction of sampling bias, which gradually increases to 1 as the training process progresses, eventually achieving complete correction; this priority experience mechanism makes the model learning process more focused on key experiences, while maintaining the stability and generalization ability of training; Step 502, Design a dynamic exploration strategy In the early stages of training, a larger exploration rate is set to encourage exploration, so that the agent can quickly learn environmental feature information; As training progresses, the value of ∈ will gradually decay, and the decay formula is as follows: ∈ t+1 =max(∈ min ,∈ t ·∈ decay ), where ∈ decay Represents the attenuation coefficient, ∈ min This represents the minimum exploration rate, ensuring that the agent's performance eventually stabilizes at a certain strategy. Meanwhile, the k value of the priority experience pool is gradually increased to 1.0 during training to improve the fairness of sample sampling and learning stability. Step 503, construct the agent agent; In the agent's policy network design, the Dueling DQN architecture is adopted to more finely distinguish state value and action advantage. The network structure consists of three parts: feature extractor, value stream, and advantage stream. The feature extractor is a multi-dimensional feature extractor constructed in step 300, which is used to extract deep semantic features from the feature data after the sliding window processing in step 105. The value stream predicts the current value V(s) through a multi-layer neural network. The advantage stream estimates the relative advantage A(s,a) of each action through a parallel network. Finally, the Q value is fused in the following way: Q(s,a)=V(s)+A(s,a), where Q(s,a) represents the Q value of choosing action a in the current state s. The state value function V(s) is used to evaluate the overall good or bad in state s, which is independent of the action taken. The advantage function A(s,a) represents the evaluation of the advantage of a specific action a taken in state s over other actions. Step 504: Construct a multi-agent transaction model; First, based on the multi-dimensional feature extractor constructed in step 300, each agent is trained independently using the training set processed by the sliding window in step 105 to ensure the independence of each agent. The training process takes place in the market environment constructed in step 400. Simultaneously, the reward function defined in step 402 is used for evaluation at each time step, and continuous improvement is made. When the profit or loss reaches the dynamic threshold set in step 403, a dynamic stop-loss mechanism is triggered, and a closing operation is performed. Then, the state environment information of each agent in the validation set portion provided by the multi-dimensional feature extractor in step 105 is used to verify the training effect of each agent. The model parameters with the best performance for each agent are selected as the model parameters for that agent. Next, all trained and validated agents are applied to the constructed trading environment. At each time step t, they independently perceive the market state and make decisions. Subsequently, the actions of multiple strategies are aggregated through a voting mechanism, and the optimal decision is finally selected for execution. Specifically, assuming there are N agents in total, the i-th agent perceives the current state s... t The action selection is a i Then, the voting frequency of all actions will be counted: in This is an indicator function that returns whether the actions are the same; if there are cases where the votes are the same, then the selection is weighted according to the average Q value of the individual policies; this mechanism can effectively improve the robustness and generalization ability of the agent's decision-making.

7. An adaptive multi-agent stock trading device based on three-state market perception, including: At least one processor, and a memory communicatively connected to said at least one processor; The memory stores instructions executable by the processor, which are executed by the at least one processor to cause the at least one processor to perform the adaptive multi-agent stock trading method based on three-state market perception as described in any one of claims 1-6.

Images