Dynamic investment decision-making method, system and equipment based on reinforcement learning and medium

Abstract

The invention relates to a dynamic investment decision-making method, system and device based on reinforcement learning and a medium, and the method comprises the steps: constructing a risk infection intensity matrix through real-time analysis of cross-market quotation data, and dynamically recognizing a risk conduction path among stocks, foreign exchange and bulk commodity markets; quantizing a node risk value based on a directed graph model and generating an asset weight adjustment amount to realize immediate blocking of a high-risk path; designing isolation effect award function feedback strategy optimization, and strengthening the cross-market risk isolation capability; an adversarial sample training strategy network is generated in combination with historical extreme event features, and the decision robustness under the black swan event is improved; and finally, outputting an executable portfolio adjustment instruction through a dynamic decision engine. According to the method, the defects of risk conduction identification lag, isolation mechanism deficiency and insufficient strategy robustness in the prior art are overcome, and real-time blocking of a cross-market risk infection chain and active defense of asset combination under extreme events are realized.

Description

Technical Field

[0001] This invention relates to the field of financial intelligent decision-making technology, specifically to a dynamic investment decision-making method, system, device, and medium based on reinforcement learning. Background Technology

[0002] In the field of financial investment decision-making, reinforcement learning technology is widely used because it can achieve dynamic strategy optimization through environmental interaction. This method learns to maximize investment returns and control risk in complex market environments by constructing a continuous feedback mechanism between the agent and the market. Especially in cross-market portfolio management, reinforcement learning models can adjust asset weights based on real-time market data to cope with the interconnected fluctuations of multiple markets such as stocks, foreign exchange, and commodities. However, when extreme events trigger a chain reaction of cross-market risks, traditional reinforcement learning decision-making frameworks face fundamental challenges.

[0003] The existing technology has the following key defects: First, the risk transmission identification is lagging: existing models rely on a single market volatility indicator and cannot capture cross-market risk contagion paths in real time (such as a stock market crash triggering a foreign exchange liquidity crunch), resulting in a delay in risk blocking; second, the isolation mechanism is lacking: traditional methods only constrain local risks through volatility and lack the ability to dynamically isolate cross-market risk transmission chains, which can easily lead to systemic collapse in extreme events; third, the strategy is not robust enough: decision models trained on historical data are difficult to cope with unexperienced black swan event patterns and fail when sudden market state changes occur. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide a dynamic investment decision-making method, system, device and medium based on reinforcement learning that can block cross-market risk contagion chains in real time and improve the robustness of decision-making in extreme scenarios.

[0005] The objective of this invention is achieved through the following solution:

[0006] In a first aspect, the present invention provides a dynamic investment decision-making method based on reinforcement learning, comprising the following steps:

[0007] S1: Perform dynamic contagion strength modeling on real-time cross-market market data obtained from the financial market data interface, calculate cross-market volatility sensitivity, liquidity difference attenuation term and Granger causality label, and generate risk contagion strength matrix;

[0008] S2: Block high-risk paths in the risk contagion intensity matrix, identify key transmission paths based on the directed graph and apply node in-degree risk constraints to generate asset weight adjustment amounts;

[0009] S3: Quantify the isolation effect of the asset weight adjustment and the risk contagion intensity matrix, calculate the product penalty term of the adjusted asset weight on the risk transmission edge and the trace operation gain term of the risk contagion intensity matrix and the adjustment matrix, and sum them up by weight to generate the isolation effect reward.

[0010] S4: Optimize the strategy adversarial approach for the risk contagion intensity matrix and the isolation effect reward. Generate perturbation samples by combining the historical extreme event features obtained from the preset historical extreme event dataset and generate strategy network parameters by minimizing the difference between the real decision and the adversarial decision through gradient optimization.

[0011] S5: Dynamic blocking decision execution is performed based on strategy network parameters. Real-time cross-market data is processed through the strategy network to generate asset weight adjustment amounts and portfolio adjustment instructions. The portfolio adjustment instructions are used to indicate the redistribution of asset portfolio weights.

[0012] In one embodiment, S1 of the dynamic investment decision-making method based on reinforcement learning provided by the present invention specifically includes the following steps:

[0013] S11: Perform dynamic conditional correlation analysis on real-time cross-market market data obtained from the financial market data interface, calculate the volatility covariance of the stock, foreign exchange and commodity markets, and generate a correlation coefficient matrix;

[0014] S12: The correlation coefficient matrix is ​​processed to calculate the contagion strength factor. The strength of inter-market transmission is quantified by the partial derivative of volatility with respect to return and the liquidity difference decay function to generate the risk contagion strength factor.

[0015] S13: Perform three-dimensional matrix processing on the risk contagion intensity factor to construct a risk contagion intensity matrix that includes the transmission relationship between the stock, foreign exchange and commodity markets.

[0016] In one embodiment, S2 of the dynamic investment decision-making method based on reinforcement learning provided by the present invention specifically includes the following steps:

[0017] S21: Perform dynamic threshold filtering on the risk contagion intensity matrix, extract cross-market transmission relationship edges that exceed the preset risk threshold, and generate a set of high-risk transmission edges;

[0018] S22: Perform directed graph structure construction on the set of high-risk transmission edges, connect stock market nodes, foreign exchange market nodes and commodity market nodes to form a three-dimensional risk transmission path network, generate a directed graph of risk transmission, and calculate the sum of the in-edge contagion strength of each node to generate the node in-degree risk value.

[0019] S23: Process the current asset weights and market volatility characteristics obtained from the risk transmission directed graph using a strategy network, and combine real-time return and volatility data to map the weight adjustment scheme and generate the original asset weight adjustment amount;

[0020] S24: Apply risk isolation constraints to the original asset weight adjustment amount, and proportionally compress the increase of high-risk asset weight based on the node in-degree risk value to generate the constrained adjustment amount;

[0021] S25: Perform capital conservation normalization on the constrained adjustment amount, scale the adjustment amount of each asset to keep the total weight at 1, and generate the asset weight adjustment amount.

[0022] In one embodiment, S3 of the dynamic investment decision-making method based on reinforcement learning provided by the present invention specifically includes the following steps:

[0023] S31: Perform combined weight update processing on the asset weight adjustment amount, add the current holding weight to generate the adjusted ratio, and generate the adjusted weight vector;

[0024] S32: Perform risk exposure calculation on the adjusted weight vector, perform the product operation of the corresponding node weights on the risk transmission edge to quantify cross-market exposure, and generate risk exposure penalty terms;

[0025] S33: Perform matrix trace operation on the risk contagion intensity matrix and asset weight adjustment amount, construct the adjustment amount diagonal matrix and perform trace operation with the contagion intensity matrix, and synthesize the isolation effect reward based on the risk exposure penalty term.

[0026] In one embodiment, the formula for calculating the isolation effect reward of the dynamic investment decision-making method based on reinforcement learning provided by the present invention is as follows:

[0027]

[0028] in, As a reward for the effectiveness of isolation, For risk exposure penalty weighting coefficient, For the risk transmission edge set, The intensity of risk contagion from market i to j These are the adjusted asset weights for market i and j, respectively. For isolation gain weighting coefficients, Represents the matrix trace operation. This is a risk-transmission intensity matrix. It is a diagonal matrix with the asset weight adjustment as the diagonal element.

[0029] In one embodiment, S4 of the dynamic investment decision-making method based on reinforcement learning provided by the present invention specifically includes the following steps:

[0030] S41: Extract features from a pre-defined historical extreme event dataset, extract market volatility mutation features and liquidity depletion feature patterns, and generate a black swan event feature vector.

[0031] S42: Generate adversarial perturbations for the risk contagion intensity matrix and black swan event feature vectors. Learn the distribution patterns of extreme events and construct matrix offsets through generative adversarial networks to generate adversarial sample matrices.

[0032] S43: Based on the adversarial sample matrix and the risk contagion intensity matrix, the policy difference is minimized, the Euclidean distance gradient of the output vector of the policy network in the real environment and the adversarial environment is calculated, the policy network parameter update amount is generated, and the policy network parameters are updated.

[0033] In one embodiment, S5 of the dynamic investment decision-making method based on reinforcement learning provided by the present invention specifically includes the following steps:

[0034] S51: Load the decision model into the policy network parameters, initialize the reinforcement learning policy network architecture and configure the weight parameters to generate a real-time decision engine.

[0035] S52: Perform feature vectorization processing on real-time cross-market market data obtained from the financial market data interface, integrate volatility, return and liquidity indicators of stock, foreign exchange and commodity markets, and generate input vectors for the strategy network;

[0036] S53: Generate weight adjustment values ​​for the input vector of the strategy network, calculate the weight changes of each asset through forward propagation of the real-time decision engine, and generate executable portfolio adjustment instructions.

[0037] Secondly, the present invention provides a dynamic investment decision-making system based on reinforcement learning, which is configured with the following modules:

[0038] The risk contagion intensity modeling module is used to dynamically model the contagion intensity of real-time cross-market market data obtained from the financial market data interface, calculate the cross-market volatility sensitivity, liquidity difference attenuation term and Granger causality label, and generate a risk contagion intensity matrix.

[0039] The high-risk path blocking module is used to block high-risk paths in the risk contagion intensity matrix. It identifies key transmission paths based on a directed graph and applies node in-degree risk constraints to generate asset weight adjustment amounts.

[0040] The isolation effect quantification module is used to quantify the isolation effect of asset weight adjustment and risk contagion intensity matrix. It calculates the product penalty term of the adjusted asset weight on the risk transmission edge and the trace operation gain term of the risk contagion intensity matrix and the adjustment matrix, and then sums them up by weight to generate the isolation effect reward.

[0041] The strategy adversarial optimization module is used to optimize the risk contagion intensity matrix and the isolation effect reward. It generates perturbation samples by combining the historical extreme event features obtained from the preset historical extreme event dataset and generates strategy network parameters by minimizing the difference between the real decision and the adversarial decision through gradient optimization.

[0042] The dynamic blocking decision execution module is used to execute dynamic blocking decisions based on strategy network parameters. It processes real-time cross-market data through the strategy network to generate asset weight adjustment amounts and generate portfolio adjustment instructions. The portfolio adjustment instructions are used to indicate the redistribution of asset portfolio weights.

[0043] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above-mentioned dynamic investment decision-making methods based on reinforcement learning.

[0044] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned dynamic investment decision-making methods based on reinforcement learning.

[0045] In summary, the reinforcement learning-based dynamic investment decision-making method provided in this application constructs a dynamic risk contagion strength matrix, which can capture cross-market volatility sensitivity, liquidity decay, and causal transmission relationships in real time, achieving millisecond-level identification of risk transmission paths between stock, foreign exchange, and commodity markets. By quantifying the in-degree risk value of nodes using a directed graph model and applying weight constraints, it can proactively cut off the risk spillover chain from high-risk markets to low-risk markets, thereby blocking the chain transmission of extreme risks across markets. The isolation effect reward function dynamically weights the risk exposure penalty term and the trace operation gain term, enabling quantitative feedback on the effectiveness of risk blocking measures. By combining historical black swan event characteristics to generate adversarial sample training strategy networks, the decision model can continuously adapt to unexperienced extreme market scenarios, improving the system's robustness in sudden crises such as liquidity mutations. Finally, through a closed-loop decision engine, it dynamically outputs asset reallocation instructions, enabling adaptive adjustment of the combined weights according to the risk transmission situation.

[0046] The method provided in this application systematically solves the problems of delayed risk identification, lack of isolation mechanisms, and vulnerability of strategies in traditional technologies, achieving a triple technical effect of real-time perception, proactive defense, and continuous evolution of cross-market risk contagion chains, and providing a dynamic safety barrier for asset portfolios under extreme events.

[0047] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0048] Figure 1 A flowchart illustrating a dynamic investment decision-making method based on reinforcement learning, provided for an embodiment of this application;

[0049] Figure 2 A schematic diagram illustrating the process of generating isolation effect rewards provided in an embodiment of this application;

[0050] Figure 3 This is a schematic diagram of the structure of a dynamic investment decision system based on reinforcement learning, provided for another embodiment of this application. Detailed Implementation

[0051] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0053] In one embodiment, such as Figure 1 As shown, a dynamic investment decision-making method based on reinforcement learning is provided. This embodiment illustrates the method's application to a terminal, but it is understood that the method can also be applied to a server, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0054] S1: Perform dynamic contagion strength modeling on real-time cross-market market data obtained from the financial market data interface, calculate cross-market volatility sensitivity, liquidity difference attenuation term and Granger causality label, and generate risk contagion strength matrix.

[0055] Specifically, the system collects real-time data from multiple markets through standardized financial data interfaces. The collected data dimensions cover core market indicators for the stock market, exchange rate and trading volume indicators for the foreign exchange market, futures contracts and open interest indicators for the commodity market, and yield and spread indicators for the bond market. The system performs data collection operations at a fixed frequency to ensure data timeliness. After collection, the system performs a preprocessing procedure on the raw data. First, missing data is processed by filling in gaps using specific methods. Then, abnormal data exceeding normal fluctuation ranges is removed. Finally, standardization processing unifies the dimensions of data across different dimensions, eliminating data interference for subsequent indicator calculations.

[0056] Furthermore, the system constructs corresponding models for the return series of individual markets, estimates the conditional volatility of individual markets through these models, and then calculates the dynamic correlation coefficient between markets based on the conditional volatility of each market. Finally, the cross-market volatility sensitivity is obtained by multiplying the conditional volatility and the dynamic correlation coefficient, which quantifies the time-varying linkage of volatility between markets. When calculating the liquidity difference decay term, the system first quantifies the liquidity index of a single market using a specific formula, and then calculates the decay term based on the liquidity difference between two markets and the time factor. This decay term exhibits a corresponding change pattern with the change in liquidity difference between markets, and also shows an exponential decay characteristic over time, consistent with the natural law of liquidity transmission.

[0057] Furthermore, the system constructs a vector autoregressive model and, by determining the lag order of the model, performs statistical tests on the standardized return series of different markets. Based on the test results, it determines whether a one-way transmission relationship exists between markets, and then generates Granger causality indicators. The system constructs a risk contagion strength matrix based on the calculation results of the above three indicators. The matrix dimension is determined by the number of cross-market asset classes. Each element in the matrix is ​​the product of the volatility sensitivity, liquidity difference attenuation term, and Granger causality indicator between the corresponding markets. The system updates the matrix in real time at the same frequency as data collection and stores the matrix in a specific format to optimize subsequent calculation efficiency.

[0058] S2: Block high-risk paths in the risk contagion intensity matrix, identify key transmission paths based on the directed graph and apply node in-degree risk constraints to generate asset weight adjustment amounts.

[0059] Specifically, the system constructs a directed risk transmission graph based on a risk contagion intensity matrix. Nodes in the directed graph correspond to various asset markets. The existence of a directed edge is determined by whether the contagion intensity between the corresponding markets meets a set condition. The weight of the directed edge directly adopts the risk contagion intensity value between the corresponding markets. The system uses a specific algorithm to screen key transmission paths. First, it calculates the importance metric for each path, which is the product of the contagion intensities of all directed edges on the path. Then, it calculates the risk transmission influence score for each node using a specific algorithm. Based on the score, key risk nodes are determined, and the directed edges between key risk nodes constitute a set of key transmission paths.

[0060] Furthermore, the system sets node in-degree risk constraints, calculates the in-degree risk of each node, which is the weighted sum of the contagion strength of all incident edges and the corresponding asset weights. Then, based on the maximum in-degree risk of all nodes, a dynamic constraint threshold is determined for each node. The dynamic constraint threshold is related to the node's own in-degree risk and the overall maximum in-degree risk. The system clearly defines the constraints that node in-degree risk must satisfy. If a node's in-degree risk violates the constraints, the asset weight adjustment process is initiated. The system constructs an optimization objective function, with the smoothness of asset weight adjustment as the optimization objective. Node in-degree constraints, a total asset weight sum of 1, and non-negative individual asset weights are used as constraints. A specific optimization algorithm transforms the constrained optimization problem into an unconstrained optimization problem, and then uses relevant mathematical conditions to solve for the optimal asset weight adjustment amount. The optimization algorithm selected by the system must ensure the efficiency of the solution process, ensuring that the calculation of the weight adjustment amount meets the time requirements of real-time decision-making.

[0061] S3: Quantify the isolation effect of the asset weight adjustment and the risk contagion intensity matrix, calculate the product penalty term of the adjusted asset weight on the risk transmission edge and the trace operation gain term of the risk contagion intensity matrix and the adjustment matrix, and sum them up by weight to generate the isolation effect reward.

[0062] Specifically, the system calculates a product penalty term for each directed edge in the set of key transmission paths. The calculation involves multiplying the adjusted asset weights of the two corresponding markets by the risk contagion strength of that directed edge. The system sums the calculation results for all directed edges in the key transmission paths to obtain the product penalty term. The magnitude of this indicator directly reflects the blocking effect of risk transmission on the key paths; a larger value indicates a weaker blocking effect. The system constructs a product matrix of the risk contagion strength matrix and the weight adjustment matrix. The weight adjustment matrix is ​​a diagonal matrix, with diagonal elements representing the weight adjustments of each asset and off-diagonal elements being 0. The system performs a trace operation on this product matrix, extracting the sum of the diagonal elements as the trace operation gain term. This gain term reflects the optimization effect of asset weight adjustments on the overall risk contagion strength. If the weight adjustment direction aligns with the risk suppression objective, the trace operation gain term will show a corresponding trend.

[0063] Preferably, the system sets two weighting coefficients, the sum of which is 1. The weighting coefficients are determined through cross-validation to ensure the rationality of the isolation effect evaluation. The system inverts the product penalty term and then performs a weighted summation with the trace operation relative gain rate to obtain the initial isolation effect reward. The trace operation relative gain rate is calculated using the trace operation baseline value before and after adjustment and the current trace operation value, ensuring the dimensionlessness of the indicator. The system normalizes the initial isolation effect reward using a specific function, mapping the reward value to a fixed interval, enhancing the distinguishability of the reward signal, and providing clear and effective feedback signals for subsequent policy network optimization.

[0064] S4: Optimize the strategy adversarial approach for the risk contagion intensity matrix and isolation effect reward. Generate perturbation samples by combining the historical extreme event features obtained from the preset historical extreme event dataset and generate strategy network parameters by minimizing the difference between the real decision and the adversarial decision through gradient optimization.

[0065] Specifically, the system collects major financial extreme events globally, covering market volatility events of different types and historical periods. For each extreme event, the system extracts multi-dimensional features, including features related to event persistence, market volatility, liquidity changes, cross-market contagion, and key transmission markets, forming a feature vector with unified dimensions. The system divides the dataset composed of all feature vectors into training and validation sets proportionally, performs standardization processing on the dataset, and transforms all feature vectors into standardized feature vectors with unified dimensions by calculating the mean and standard deviation of the features, providing high-quality data support for model training.

[0066] The system constructs an extreme event feature generation model based on a variational autoencoder, which consists of an encoder and a decoder. The encoder employs a multi-layer fully connected network structure, receiving standardized feature vectors as input and outputting the mean and variance vectors of the latent variables. The decoder also employs a multi-layer fully connected network structure, receiving latent variables as input and outputting reconstructed feature vectors. When training this generative model, the system aims to minimize the sum of the reconstruction loss and the KL divergence. The reconstruction loss uses mean squared error loss, and the KL divergence is used to measure the difference between the encoder's output distribution and the standard normal distribution. After training, the system generates perturbation samples by sampling the latent variables, while simultaneously applying strength constraints to the perturbation samples to ensure that the perturbation samples both cover variations of extreme scenarios that have not occurred before and do not deviate from the core features of extreme events.

[0067] The system employs an Actor-Critic reinforcement learning architecture to design the policy network. The Actor network is responsible for outputting asset weight adjustments, with inputs including real-time market features, risk contagion strength matrix features, and isolation effect rewards. The network structure includes an input layer, multiple hidden layers, and an output layer. The hidden layers use specific activation functions, while the output layer uses the Softmax activation function to ensure that the output weight adjustments satisfy non-negativity and normalization constraints. The Critic network is responsible for evaluating state value, with inputs including the input features of the Actor network and the output weight adjustments. Its network structure also includes an input layer, multiple hidden layers, and an output layer. The output layer is a single neuron using a linear activation function, outputting the state value.

[0068] The system constructs an adversarial training objective function, which includes a difference term between the real decision and the adversarial decision, and a term related to the isolation effect reward. The real decision is the asset weight adjustment based on the characteristics of the original extreme events, while the adversarial decision is the asset weight adjustment based on the perturbation samples. The isolation effect reward term consists of the product of the isolation effect reward and the action value function, which is estimated by the Critic network. The system uses a specific optimization algorithm to optimize the objective function, setting relevant parameters during training, including learning rate, batch size, and number of iterations. A fixed number of iterations are performed at each interval. The system evaluates the model performance on a validation set and employs an early stopping mechanism, stopping training when the validation set loss does not decrease for several consecutive rounds. Finally, the optimal policy network parameters are output.

[0069] S5: Dynamic blocking decision execution is performed based on strategy network parameters. Real-time cross-market data is processed through the strategy network to generate asset weight adjustment amounts and portfolio adjustment instructions. The portfolio adjustment instructions are used to indicate the redistribution of asset portfolio weights.

[0070] Specifically, the system receives real-time cross-market market data pushed from a financial data interface and executes a data processing flow at a fixed frequency. First, the system cleans the raw data, using specific methods to handle missing data and statistical methods to remove outliers, ensuring data integrity and accuracy. Then, the system executes a feature engineering process to extract relevant features consistent with previous indicator calculations, including volatility-related features, liquidity-related features, and causal relationship-related features. Through standardization and dimensionality compression, it generates the input feature vector required by the strategy network. The system inputs the processed feature vector into an Actor network loaded with optimal parameters, and the network outputs the initial asset weight adjustment. The system performs a post-processing flow on the initial weight adjustment. First, it imposes amplitude constraints, limiting the single-time weight adjustment range for a single asset; adjustments exceeding the constraints are reduced proportionally. Second, it performs normalization to ensure that the sum of all adjusted asset weights is 1. Finally, it performs risk control verification, checking whether the adjusted asset weights meet preset risk control threshold requirements, including single asset weight thresholds and total weight thresholds for high-risk market assets. If not, a second adjustment is triggered until the weights meet all risk control requirements.

[0071] Preferably, the system can generate portfolio adjustment instructions using a standardized financial transaction protocol format, ensuring compatibility between different trading platforms. The adjustment instruction includes multiple core fields: a basic information field uniquely identifies the instruction, records its generation time and execution priority; an asset information field specifies the asset class code and the code of the specific underlying asset; an adjustment information field details the asset weights before and after the adjustment, the adjustment direction, and the adjustment ratio; and a risk control information field sets safety constraints such as slippage limits, execution time limits, and stop-loss thresholds.

[0072] Furthermore, the system sends portfolio adjustment instructions to the trading system via a standardized protocol interface. After executing the instructions, the trading system returns transaction confirmation information to the system, including key data such as transaction price, volume, time, and slippage rate. The system stores this feedback data in a time-series database and updates the asset weights to the weights after the actual transaction. These updated weights will serve as historical weight data for the next decision-making process, forming a closed-loop decision-making mechanism of "data acquisition - feature processing - decision generation - instruction execution - feedback update." This ensures the system's real-time adaptability to market changes and guarantees the continuity and effectiveness of dynamic investment decisions.

[0073] In summary, the reinforcement learning-based dynamic investment decision-making method provided in this application constructs a dynamic risk contagion strength matrix, which can capture cross-market volatility sensitivity, liquidity decay, and causal transmission relationships in real time, achieving millisecond-level identification of risk transmission paths between stock, foreign exchange, and commodity markets. By quantifying the in-degree risk value of nodes using a directed graph model and applying weight constraints, it can proactively cut off the risk spillover chain from high-risk markets to low-risk markets, thereby blocking the chain transmission of extreme risks across markets. The isolation effect reward function dynamically weights the risk exposure penalty term and the trace operation gain term, enabling quantitative feedback on the effectiveness of risk blocking measures. By combining historical black swan event characteristics to generate adversarial sample training strategy networks, the decision model can continuously adapt to unexperienced extreme market scenarios, improving the system's robustness in sudden crises such as liquidity mutations. Finally, through a closed-loop decision engine, it dynamically outputs asset reallocation instructions, enabling adaptive adjustment of the combined weights according to the risk transmission situation.

[0074] The method provided in this application systematically solves the problems of delayed risk identification, lack of isolation mechanisms, and vulnerability of strategies in traditional technologies, achieving a triple technical effect of real-time perception, proactive defense, and continuous evolution of cross-market risk contagion chains, and providing a dynamic safety barrier for asset portfolios under extreme events.

[0075] In one embodiment, S1 of the dynamic investment decision-making method based on reinforcement learning provided by the present invention specifically includes the following steps:

[0076] S11: Perform dynamic conditional correlation analysis on real-time cross-market market data obtained from the financial market data interface, calculate the volatility covariance of the stock, foreign exchange and commodity markets, and generate a correlation coefficient matrix.

[0077] Specifically, the system acquires real-time market data for stocks, foreign exchange, and commodities through financial market data interfaces. The data dimensions cover stock market data such as trading prices, trading volume, and turnover frequency; foreign exchange market data such as currency pair exchange rates, bid-ask spreads, and transaction amounts; and commodity market data such as futures contract trading prices, open interest, and basis. The system performs data collection at fixed intervals to ensure that the data reflects the current market state in a timely manner. After data collection, the system initiates a preprocessing process. First, missing data is filled in using a method consistent with data trends to ensure that the filled data does not deviate from actual market characteristics. Then, outlier data exceeding the normal data distribution range is removed to prevent outliers from interfering with subsequent analysis results. Finally, standardization processing eliminates dimensional differences between different data types, providing a unified comparison benchmark for all data.

[0078] Furthermore, the system performs dynamic conditional correlation analysis on the preprocessed market data. First, it calculates the return series for each market, deriving returns from price data at adjacent time points to form a continuous return series. Based on the return series, the system calculates volatility covariance. It first estimates the time-varying volatility of each market's return series using a specific model that captures the volatility clustering characteristics of the return series, accurately reflecting the dynamic changes in market volatility. Then, based on the time-varying volatility of each market, it calculates the dynamic covariance between markets. The covariance calculation is based on the volatility linkage between different market return series, quantifying the degree of volatility coordination between two markets on the same time dimension. Based on the calculated volatility covariance, the system constructs a correlation coefficient matrix. The matrix dimension is determined by the number of market categories involved in the analysis. Each element in the matrix is ​​the correlation coefficient between two corresponding markets. The correlation coefficient is calculated as the ratio of the covariance to the volatility of the two corresponding markets, used to characterize the strength of the volatility linkage between the two markets.

[0079] S12: The correlation coefficient matrix is ​​processed by calculating the contagion strength factor. The strength of inter-market transmission is quantified by the partial derivative of volatility with respect to return and the liquidity difference decay function, and the risk contagion strength factor is generated.

[0080] Specifically, the system uses the correlation coefficient matrix as basic data to calculate the contagion strength factor and the partial derivative of volatility with respect to returns. This partial derivative quantifies the sensitivity of volatility to changes in returns, reflecting the magnitude of volatility's response to small changes in returns. During the calculation, the system derives the partial derivative value through calculus operations based on the return series and corresponding volatility series of each market. This value reflects the intrinsic correlation between market volatility and return changes, providing a fundamental parameter for quantifying the strength of inter-market transmission.

[0081] The system calculates a liquidity difference decay function. First, it quantifies the liquidity indicator of a single market using a specific formula. This indicator, constructed based on market trading volume and price fluctuation data, objectively reflects the market's liquidity level. Then, it calculates the liquidity difference between any two markets, representing the absolute difference between their liquidity indicators, thus characterizing the gap in liquidity levels. The system constructs a decay function based on this liquidity difference. The function's value follows a corresponding pattern as the liquidity difference increases; as the liquidity difference between the two markets increases, the function value decreases, conforming to the natural decay characteristics of liquidity transmission and quantifying the inhibitory effect of liquidity differences on inter-market transmission.

[0082] Furthermore, the system combines the partial derivative of volatility with respect to returns with the calculation results of the liquidity difference decay function to quantify the strength of risk transmission between markets. In this combination, the system uses the correlation coefficients in the correlation coefficient matrix as a basis, and adjusts the correlation coefficients using the partial derivative values ​​and the decay function values ​​as correction factors. The adjusted result is the risk contagion strength factor. This factor comprehensively considers the volatility linkage between markets, the sensitivity of volatility to returns, and the decaying effect of liquidity differences, enabling a comprehensive and accurate quantification of the strength level of risk transmission between different markets.

[0083] S13: Perform three-dimensional matrix processing on the risk contagion intensity factor to construct a risk contagion intensity matrix that includes the transmission relationship between the stock, foreign exchange and commodity markets.

[0084] Specifically, after receiving the calculation results of the risk contagion intensity factor, the system initiates a three-dimensional matrix processing flow. The system first defines the dimensions of the three-dimensional matrix, with the three dimensions corresponding to the stock market, foreign exchange market, and commodity market, respectively. Each dimension of the matrix includes all asset classes or sub-markets of the corresponding market, ensuring that the matrix can fully cover all potential transmission paths between the three markets. The system assigns each risk contagion intensity factor to a specific dimension and determines its exact position index in the three-dimensional matrix based on the two market categories it corresponds to, ensuring that each factor is accurately mapped to its corresponding element position in the matrix.

[0085] The system constructs a three-dimensional risk contagion strength matrix. Each element in the matrix corresponds to the risk contagion strength between two different markets, and the element value is directly taken from the corresponding risk contagion strength factor value. The row, column, and depth dimensions of the matrix correspond to different market types, clearly presenting the bidirectional transmission relationships between stocks and foreign exchange, stocks and commodities, and foreign exchange and commodities through a three-dimensional structure. It also includes the transmission relationships between different asset classes or sub-markets within the same market. During the matrix construction process, the system verifies the position mapping of all factors to ensure no factors are omitted or incorrectly positioned, guaranteeing the integrity and accuracy of the matrix.

[0086] Preferably, the system standardizes the format of the constructed three-dimensional risk contagion intensity matrix, adopting a specific format that supports efficient storage and computation of three-dimensional data, thus optimizing the matrix's read / write speed and subsequent computational efficiency. The system updates the three-dimensional matrix at a fixed period, with the update frequency consistent with the data acquisition, correlation coefficient matrix update, and contagion intensity factor calculation cycles, ensuring that the matrix can reflect the dynamic changes in risk contagion intensity between markets in real time. This three-dimensional risk contagion intensity matrix comprehensively covers all transmission relationships between the stock, foreign exchange, and commodity markets, providing structured core data support for subsequent steps such as high-risk path blocking and isolation effect quantification, ensuring the orderly conduct of the entire dynamic investment decision-making process.

[0087] In one embodiment, S2 of the dynamic investment decision-making method based on reinforcement learning provided by the present invention specifically includes the following steps:

[0088] S21: Perform dynamic threshold filtering on the risk contagion intensity matrix, extract cross-market transmission relationship edges that exceed the preset risk threshold, and generate a set of high-risk transmission edges.

[0089] Specifically, the system receives a three-dimensional risk contagion intensity matrix, which comprehensively covers the risk transmission intensity data between and within the stock, foreign exchange, and commodity markets. The system first preprocesses the matrix, traversing all elements to identify the subject and object of the transmission relationship for each element, clarifying the bidirectional nature of the transmission relationship, and ensuring that no key transmission paths are missed in subsequent screening processes. The system then activates a dynamic threshold screening mechanism. The threshold is set based on the distribution characteristics of historical risk transmission intensity data and the current overall market risk level. By analyzing the extreme values, average values, and fluctuation ranges of transmission intensity under different market conditions in historical periods, combined with the current real-time market volatility characteristics, liquidity levels, and macroeconomic market environment indicators, the system dynamically determines the risk threshold.

[0090] Furthermore, the system compares the strength factor corresponding to each transmission relationship in the matrix with a dynamic threshold, extracting all transmission relationships whose strength factors exceed the threshold. These transmission relationships are the cross-market transmission relationship edges with high-risk transmission characteristics. The system annotates the extracted high-risk transmission relationship edges with attributes including the starting market type, ending market type, starting asset sub-category, and ending asset sub-category, ensuring that the transmission path of each edge is clear and traceable. The system performs deduplication verification on the annotated high-risk transmission relationship edges, removing duplicate records to avoid data redundancy in subsequent processing. Finally, the system integrates all the filtered, annotated, and verified high-risk transmission relationship edges to generate a high-risk transmission edge set.

[0091] S22: Perform directed graph structure construction on the set of high-risk transmission edges, connect stock market nodes, foreign exchange market nodes and commodity market nodes to form a three-dimensional risk transmission path network, generate a directed risk transmission graph, and calculate the sum of the in-edge contagion strength of each node to generate the node in-degree risk value.

[0092] Specifically, the system initiates a directed graph structure construction process based on a set of high-risk transmission edges. The system first defines the node system of the directed graph, categorizing nodes into three main types according to market type: stock market nodes, foreign exchange market nodes, and commodity market nodes. Each type of node is further subdivided into specific asset classes or sub-market levels to ensure that nodes accurately correspond to the asset entities in the actual market. Based on the start and end points of each edge in the high-risk transmission edge set, the system connects the corresponding nodes to construct a three-dimensional risk transmission path network. During the connection process, the system strictly adheres to the directional characteristics of the transmission relationships, ensuring that the direction of the directed edges is consistent with the direction of risk transmission, accurately presenting the bidirectional transmission paths between stocks and foreign exchange, stocks and commodities, and foreign exchange and commodities, while fully preserving the transmission relationships between different asset classes or sub-markets within the same market.

[0093] After the directed graph is constructed, the system calculates the sum of the ingress edge contagion strengths of each node to generate the node's ingress degree risk value. The system traverses each node in the directed graph, identifying all directed edges ending at that node, i.e., the node's ingress edges. For each ingress edge, the system extracts its corresponding risk contagion strength factor, sums the strength factors of all ingress edges, and obtains the total ingress edge contagion strength of that node. This sum is the node's ingress degree risk value. During the calculation process, the system performs a secondary verification of the ingress edge identification for each node to ensure no ingress edges are missed or misjudged, guaranteeing the accuracy of the node's ingress degree risk value calculation. The system associates and stores the identification information of all nodes with their corresponding ingress degree risk values, forming a node ingress degree risk value dataset.

[0094] S23: Process the current asset weights and market volatility characteristics obtained from the risk transmission directed graph using a strategy network, and combine real-time return and volatility data to map the weight adjustment scheme and generate the original asset weight adjustment amount.

[0095] Specifically, the system acquires the weight allocation data of the current asset portfolio, reflecting the proportion of various assets in the portfolio. Simultaneously, it collects real-time volatility characteristic data from the stock, foreign exchange, and commodity markets, covering indicators related to market price trends, volatility frequency, and volatility amplitude. The system integrates the current asset weight data, market volatility characteristic data, real-time return data, and volatility data to form the input feature set for the strategy network. During the construction of the input feature set, the system ensures consistency in the time dimension of various data types, achieving accurate matching of multi-dimensional data at the same point in time and avoiding decision-making biases caused by data time misalignment.

[0096] The system inputs the integrated feature set into a pre-defined policy network. This network, based on a reinforcement learning architecture, is capable of processing high-dimensional financial data. The network first extracts and transforms the dimensions of the input features. Through hidden layer operations, it fuses and abstracts multi-dimensional features, uncovering the intrinsic relationships between different features, with a focus on capturing the mapping relationship between current asset weights, market volatility characteristics, and real-time returns and volatility. Based on the learned mapping patterns and a pre-defined decision objective, the policy network generates preliminary suggestions for the direction and magnitude of weight adjustments.

[0097] Based on the adjustment suggestions output from the network, the system further maps them into specific weight adjustment plans. These plans clearly define the direction (increase or decrease) and initial adjustment magnitude for the weights of various assets. The system performs a rationality check on the generated weight adjustment plans, identifying any suggestions that clearly conflict with fundamental market principles or the inherent characteristics of the assets, ensuring the plans are practically feasible. Once the check passes, the system quantifies the weight adjustment plan into specific numerical values, generating the original asset weight adjustment amounts.

[0098] S24: Apply risk isolation constraints to the original asset weight adjustment amount, and proportionally compress the increase of high-risk asset weights based on the node in-degree risk value to generate the constrained adjustment amount.

[0099] Specifically, the system accesses the node in-degree risk value dataset and correlates the node in-degree risk value corresponding to each asset with the original weight adjustment amount of that asset, clarifying the correspondence between the risk level and adjustment range of various assets. The system classifies risk levels based on the relative level of node in-degree risk values. Assets with in-degree risk values ​​in a higher range are identified as high-risk assets. The transmission paths corresponding to these assets have strong risk diffusion potential, and risk transmission needs to be blocked by constraining the adjustment range.

[0100] The system performs proportional compression on the original weight adjustments of high-risk assets. The compression ratio is directly related to the in-degree risk value of the node corresponding to the asset; the higher the in-degree risk value, the larger the compression ratio. This ensures that the weight increase of high-risk assets is effectively controlled, reducing the possibility of further risk diffusion through such assets. During the compression process, the system maintains the relative stability of the original weight adjustments of low-risk assets, only imposing targeted constraints on the increase of high-risk assets to avoid excessive adjustments that could lead to an imbalance in the overall portfolio structure.

[0101] The system performs a secondary verification of the compressed weight adjustments, checking whether the increased weights of high-risk assets meet the risk isolation objective, and confirming that the adjustments to low-risk assets have not been unreasonably affected by the constraint processing. If the verification finds that the adjustments to some high-risk assets still do not meet the constraint requirements, the system will perform compression processing again until the weight increases of all high-risk assets meet the risk isolation constraints. Finally, the system integrates all the constrained weight adjustment data to generate the constrained adjustment amount.

[0102] S25: Perform capital conservation normalization on the constrained adjustment amount, scale the adjustment amount of each asset to keep the total weight at 1, and generate the asset weight adjustment amount.

[0103] Specifically, upon receiving the constraint adjustment amount, the system first calculates the sum of the constraint adjustments for all assets, identifying the deviation between the current total adjustment amount and the capital conservation requirement. Capital conservation requires that the total weight of the adjusted asset portfolio remain at 1; this is a fundamental principle of portfolio allocation, ensuring that no funds are idle or over-allocated. Based on the calculated deviation value, the system initiates a normalization process to determine the overall scaling ratio. The scaling ratio is calculated based on the sum of the constraint adjustments, using specific operations to obtain a scaling factor that makes the total weight equal to 1.

[0104] Furthermore, the system multiplies the constrained adjustment amount of each asset by a scaling factor, scaling the adjustment amount of each asset proportionally. During scaling, the system strictly maintains the relative proportional relationship between the adjustment amounts of each asset, ensuring that the effect of the initial risk isolation constraint processing is not affected, while ensuring that the total weight after adjustment accurately meets the capital conservation requirement. The system performs a non-negativity check on the scaled asset weights. If any adjusted weight is negative, the system will correct the adjustment amount of that asset to ensure that the weights of all assets are within a reasonable range and comply with the actual operation specifications of the investment portfolio. After the check passes, the system integrates all scaled asset adjustment amounts to generate the final asset weight adjustment amount. This adjustment amount satisfies both the risk isolation constraint requirement and the capital conservation principle, clearly defining the final adjustment range and the adjusted weight ratio of each type of asset.

[0105] In one embodiment, such as Figure 2 As shown, S3 of the dynamic investment decision-making method based on reinforcement learning provided by the present invention specifically includes the following steps:

[0106] S31: Perform combined weight update processing on the asset weight adjustment amount, add the current holding weight to generate the adjusted ratio, and generate the adjusted weight vector.

[0107] Specifically, the system receives asset weight adjustments and simultaneously retrieves the current portfolio's holding weight data, ensuring a consistent asset classification system for both types of data. The system performs asset dimension consistency checks by comparing asset identification information, including asset codes, market affiliation, and sub-categories, to ensure that the weight adjustment of each asset accurately matches its corresponding holding weight. If the check detects a mismatch, the system executes a data calibration process to correct the asset identification mapping until the asset dimensions of the two types of data are completely aligned. Preferably, the system performs portfolio weight update calculations, using a formula to calculate the adjusted proportions of each asset type:

[0108]

[0109] in, Let be the adjusted weight of asset class i. Let be the current holding weight of asset class i. This represents the weight adjustment for asset class i. During the calculation, the system performs the overlay operation one by one according to asset class, verifies the result of each calculation in real time, and checks for data anomalies or logical errors.

[0110] After all asset adjustments are calculated, the system arranges the adjusted proportions in an ordered manner according to the asset sorting rules corresponding to the risk contagion intensity matrix, ensuring the consistency of subsequent data processing. The system then integrates the sorted adjusted proportions into a vector form, generating an adjustment weight vector. , where n is the total number of asset classes. The system standardizes the format of the adjustment weight vector and associates the asset identifier with the position of the vector elements.

[0111] S32: Perform risk exposure calculation on the adjusted weight vector, perform the product operation of the corresponding node weights on the risk transmission edge to quantify cross-market exposure, and generate a risk exposure penalty term.

[0112] Specifically, the system calls upon a risk transmission edge set and an adjusted weight vector. The risk transmission edge set includes the starting market, ending market, and corresponding risk contagion intensity for each transmission edge. The adjusted weight vector provides the adjusted weights for various assets. The system establishes a correlation between each transmission edge and its corresponding starting and ending assets' adjusted weights through market identifier matching, ensuring unbiased data correspondence. Preferably, the system quantifies cross-market risk exposure, using a formula to calculate the quantified risk exposure value for a single transmission edge:

[0113]

[0114] in, Let this be the quantified value of the risk exposure along the transmission path from market i to market j. The risk contagion intensity from market i to market j. Let be the adjusted weight of the i-th market asset. Let be the adjusted weight of the j-th market asset. The system iterates through all risk transmission edges, calculates the quantified value for each and stores it temporarily, while also recording the corresponding transmission edge identifier.

[0115] Furthermore, the system performs a summation operation on the risk exposure quantification values ​​of all individual transmission edges, and generates a risk exposure penalty term using a formula:

[0116]

[0117] in, For risk exposure penalties, This is the risk transmission edge set. During the summation process, the system accumulates the quantified values ​​in a preset order to avoid omissions or duplicate calculations. After completion, the results are verified to ensure the accuracy of the calculation, and finally, the risk exposure penalty item data is generated.

[0118] S33: Perform matrix trace operation on the risk contagion intensity matrix and asset weight adjustment amount, construct the adjustment amount diagonal matrix and perform trace operation with the contagion intensity matrix, and synthesize the isolation effect reward based on the risk exposure penalty term.

[0119] Specifically, the system retrieves the risk contagion intensity matrix, asset weight adjustment amounts, risk exposure penalty terms, and two types of preset weight coefficients. The system constructs a diagonal matrix of adjustment amounts, with asset weight adjustment amounts as diagonal elements and off-diagonal elements set according to matrix operation specifications, ensuring that the dimensions of the diagonal matrix are consistent with the risk contagion intensity matrix.

[0120]

[0121] in, This refers to the weighting adjustments for various asset classes. Furthermore, the system performs matrix multiplication to adjust the risk contagion strength matrix. Diagonal matrix of adjustment amount Multiplying them together yields the product matrix. The system performs trace operation on the product matrix and calculates the trace result using the following formula:

[0122]

[0123] in, Let be the i-th diagonal element of the product matrix, and n be the total number of asset classes.

[0124] Preferably, the system synthesizes the isolation effect reward according to a preset formula, and the formula for calculating the isolation effect reward is:

[0125]

[0126] in, As a reward for the effectiveness of isolation, For risk exposure penalty weighting coefficient, For the risk transmission edge set, The intensity of risk contagion from market i to j These are the adjusted asset weights for market i and j, respectively. For isolation gain weighting coefficients, Represents the matrix trace operation. This is a risk-transmission intensity matrix. It is a diagonal matrix with the asset weight adjustment as the diagonal element.

[0127] In one embodiment, S4 of the dynamic investment decision-making method based on reinforcement learning provided by the present invention specifically includes the following steps:

[0128] S41: Extract features from a pre-defined historical extreme event dataset, extract market volatility mutation features and liquidity depletion feature patterns, and generate a black swan event feature vector.

[0129] Specifically, the system retrieves a pre-defined historical extreme event dataset. This dataset covers raw data on different types and periods of financial extreme events globally, including market price series, transaction data, and liquidity-related data. The system first preprocesses the dataset, standardizing the time dimension of the data, handling missing data through specific methods, and removing abnormal data that exceeds the normal distribution range to ensure data integrity and consistency.

[0130] Preferably, the system extracts the characteristics of sudden changes in market volatility, and the rate of change of volatility can be calculated using a formula:

[0131]

[0132] in, For the degree of volatility mutation in the i-th type of market, Let be the volatility of the i-th type of market at the moment of the extreme event. Let be the average volatility of market i before the occurrence of an extreme event within a specific time window, and T be the length of the feature extraction time window. The system calculates the degree of volatility mutation for each market category, quantifying the sudden changes in market volatility. The system extracts liquidity depletion feature patterns and calculates liquidity change indicators using a formula:

[0133]

[0134] in, The degree of liquidity depletion in market i. Let be the liquidity indicator for market i at the moment of an extreme event. This represents the average liquidity indicator for market i-th type over a specific time window prior to the occurrence of an extreme event. The system integrates the volatility mutation characteristics and liquidity depletion characteristics of all markets, arranging them in a preset order to form a unified-dimensional black swan event feature vector. , where n is the total number of market categories.

[0135] S42: Generate adversarial perturbations for the risk contagion intensity matrix and black swan event feature vectors. Learn the distribution patterns of extreme events and construct matrix offsets through generative adversarial networks to generate adversarial sample matrices.

[0136] Specifically, the system uses the risk contagion strength matrix and black swan event feature vectors to construct a generative adversarial network (GAN) to learn the distribution patterns of extreme events. The GAN consists of a generator and a discriminator. The generator receives a fused data set of random noise vectors and black swan event feature vectors as input and outputs the risk contagion strength matrix offset through a multi-layer network structure. The discriminator receives the risk contagion strength matrix offset corresponding to real extreme events and the offset output by the generator as input, and outputs a judgment result to distinguish the data sources.

[0137] The system trains a generative adversarial network (GAN) and iteratively optimizes the network parameters through adversarial interaction between the generator and the discriminator. The generator's training objective is to generate matrix offsets that the discriminator might misclassify as real data, while the discriminator's training objective is to accurately distinguish between real and generated offsets. During training, the system uses a cross-entropy loss function to measure the discriminator's classification error and the probability of generated data being classified as real data as the generator's loss metric. Network parameters are updated through backpropagation until training converges.

[0138] The system generates matrix offsets using a trained generator. The adversarial example matrix is ​​generated using the following formula:

[0139]

[0140] in, For the adversarial sample matrix, This is the original risk-infection intensity matrix. This is the matrix offset output by the generator. The system performs a rationality check on the generated adversarial example matrix to ensure that the matrix elements conform to the physical meaning of the risk contagion intensity and have no abnormal values. After the check passes, the system stores the adversarial example matrix, which integrates the perturbation information of extreme event characteristics and risk contagion intensity.

[0141] S43: Based on the adversarial sample matrix and the risk contagion intensity matrix, the policy difference is minimized, the Euclidean distance gradient of the output vector of the policy network in the real environment and the adversarial environment is calculated, the policy network parameter update amount is generated, and the policy network parameters are updated.

[0142] Specifically, the system loads all parameters of the current policy network, including the weights and biases of each layer. The system inputs the original risk contagion strength matrix into the policy network to obtain the asset weight adjustment output vector under real-world conditions. Simultaneously, the adversarial sample matrix is ​​input into the same policy network to obtain the asset weight adjustment output vector under adversarial conditions. .

[0143] Furthermore, the system calculates the Euclidean distance between the true output vector and the adversarial output vector, and uses a formula to quantify the strategy difference:

[0144]

[0145] Where D is the Euclidean distance and m is the dimension of the output vector. The k-th element of the output vector for the real environment. This represents the k-th element of the output vector in the adversarial environment. The magnitude of the Euclidean distance directly reflects the degree of difference in the policy network's decision output between the real and adversarial environments.

[0146] The system performs gradient calculation on the Euclidean distance and generates the gradient vector using the following formula:

[0147]

[0148] in, Euclidean distance with respect to policy network parameters gradient vector, This represents the set of all trainable parameters for the policy network. The system constructs a parameter update objective based on gradient vectors, minimizes policy differences by minimizing Euclidean distance, and generates and updates the policy network parameters using the following formula:

[0149]

[0150] in, For the updated policy network parameters, The network parameters were set before the update. The step size coefficients are updated for the parameters. The system verifies the validity of the updated parameters to ensure network stability. The updated policy network can output consistent decisions in both real and adversarial environments, thus improving policy robustness.

[0151] In one embodiment, S5 of the dynamic investment decision-making method based on reinforcement learning provided by the present invention specifically includes the following steps:

[0152] S51: Load the decision model into the policy network parameters, initialize the reinforcement learning policy network architecture and configure the weight parameters to generate a real-time decision engine.

[0153] Specifically, the system retrieves the updated policy network parameters, which include weight and bias data for each layer of the policy network. The system initiates the reinforcement learning policy network architecture initialization process, using the pre-defined Actor-Critic architecture. This process clearly defines the hierarchical structure of the input, hidden, and output layers. The input layer dimension matches the total dimension of the policy network's input features. The hidden layers employ a multi-layered network structure to achieve in-depth feature extraction. The output layer dimension is consistent with the total number of asset classes. The system then performs parameter configuration, assigning the retrieved parameters to their corresponding positions according to network layer and neuron identifiers. The entire network parameter loading is completed using the following formula:

[0154]

[0155] in, This represents the complete set of trainable parameters of the policy network. Let be the weight parameters of the k-th neuron in the l-th layer of the network. This represents the bias parameter for the k-th neuron in the l-th layer of the network. The system verifies the completeness and accuracy of the parameter configuration, ensuring that there are no missing or misaligned parameters in each layer. Simultaneously, it initializes the forward propagation logic of the network, clarifying the activation function call order and data transmission path for each layer. After the parameter configuration and computation logic initialization are complete, the system integrates the network architecture, loads the parameters, and executes the computation process to generate a real-time decision engine. This engine has the ability to quickly receive input data, perform computations, and output results.

[0156] S52: Performs feature vectorization processing on real-time cross-market market data obtained from the financial market data interface, integrates volatility, return and liquidity indicators of stock, foreign exchange and commodity markets, and generates input vectors for the strategy network.

[0157] Specifically, the system continuously acquires real-time market data for stocks, foreign exchange, and commodities through financial market data interfaces, covering volatility, yield, and liquidity indicators for each of the three markets. The system performs a preprocessing procedure on the real-time collected raw data. First, it standardizes the timestamps of all data to ensure accurate time alignment across different markets and indicators. Then, it processes missing data and removes outliers that exceed the normal distribution range, ensuring the timeliness and reliability of the data. The system then performs feature standardization on the preprocessed indicators, using formulas to transform all raw feature values:

[0158]

[0159] in, Let p be the standardized eigenvalue. Let p be the original value of the p-th feature. This is the historical statistical mean of this feature. This represents the historical statistical standard deviation of this feature. The system integrates standardized features from the three market categories according to a preset feature sorting rule, sequentially arranging the volatility, return, and liquidity features of the stock market, foreign exchange market, and commodity market to form a unified-dimensional strategy network input vector. Where q is the total number of input features. The system performs dimensionality verification on the generated input vector to confirm that its dimension completely matches the dimension of the input layer of the real-time decision engine. After the verification is passed, the input vector is transmitted to the real-time decision engine as the basis for subsequent weight adjustment calculations.

[0160] S53: Generate weight adjustment values ​​for the input vector of the strategy network, calculate the weight changes of each asset through forward propagation of the real-time decision engine, and generate executable portfolio adjustment instructions.

[0161] Specifically, the system inputs the policy network input vector into the real-time decision engine, triggering the forward propagation computation process. The input vector first passes through the input layer into the first hidden layer. After linear operations on the weight matrix and bias, it is processed by an activation function to generate a hidden layer feature vector. This feature vector flows sequentially through each subsequent hidden layer according to the preset network structure. Each layer performs linear operations and activation function processing, achieving gradual abstraction and optimization of the features, and finally transmits it to the output layer. The output layer performs linear operations on the received feature vector to obtain the raw output results of the changes in each asset weight, and then generates standardized asset weight adjustment amounts using a formula.

[0162]

[0163] in, Let be the final weight adjustment for asset class i. To output the linear output of the i-th neuron in the output layer, the Softmax function ensures that all asset weight adjustments meet the requirements of non-negativity and normalization. The system performs compliance checks on the generated weight adjustments, verifying whether they conform to the principle of capital conservation and preset risk control boundaries to avoid adjustments exceeding the scope of actual operation. After successful verification, the system converts the weight adjustments into executable portfolio adjustment instructions, which include core information such as asset class identifier, adjustment direction, adjustment magnitude, and target weight. The system encapsulates the adjustment instructions according to standardized financial transaction formats to ensure accurate recognition and parsing by the trading system, completing the entire execution process from real-time data input to decision instruction output.

[0164] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0165] Based on the same inventive concept, this application also provides a reinforcement learning-based dynamic investment decision system for implementing the aforementioned reinforcement learning-based dynamic investment decision method. The solution provided by this system is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more reinforcement learning-based dynamic investment decision system embodiments provided below can be found in the limitations of the reinforcement learning-based dynamic investment decision method described above, and will not be repeated here.

[0166] Preferably, such as Figure 3 As shown, the present invention provides a dynamic investment decision-making system 600 based on reinforcement learning, which is configured with the following modules:

[0167] The risk contagion intensity modeling module 610 is used to dynamically model the contagion intensity of real-time cross-market market data obtained from the financial market data interface, calculate the cross-market volatility sensitivity, liquidity difference attenuation term and Granger causality label, and generate a risk contagion intensity matrix.

[0168] The high-risk path blocking module 620 is used to block high-risk paths in the risk contagion intensity matrix. It identifies key transmission paths based on a directed graph and applies node in-degree risk constraints to generate asset weight adjustment amounts.

[0169] The isolation effect quantification module 630 is used to quantify the isolation effect of the asset weight adjustment amount and the risk contagion intensity matrix. It calculates the product penalty term of the adjusted asset weight on the risk transmission edge and the trace operation gain term of the risk contagion intensity matrix and the adjustment amount matrix, and then sums them up by weight to generate the isolation effect reward.

[0170] The strategy adversarial optimization module 640 is used to optimize the risk contagion intensity matrix and the isolation effect reward. It generates perturbation samples by combining the historical extreme event features obtained from the preset historical extreme event dataset and generates strategy network parameters by minimizing the difference between the real decision and the adversarial decision through gradient optimization.

[0171] The dynamic blocking decision execution module 650 is used to execute dynamic blocking decisions based on strategy network parameters. It processes real-time cross-market data through the strategy network to generate asset weight adjustment amounts and generate portfolio adjustment instructions. The portfolio adjustment instructions are used to indicate the redistribution of asset portfolio weights.

[0172] Preferably, the risk transmission intensity modeling module 610 provided in this application is configured with the following units:

[0173] The dynamic conditional correlation analysis unit is used to perform dynamic conditional correlation analysis on real-time cross-market market data obtained from the financial market data interface, calculate the volatility covariance of the stock, foreign exchange and commodity markets, and generate a correlation coefficient matrix.

[0174] The contagion strength factor calculation unit is used to calculate the contagion strength factor on the correlation coefficient matrix. It quantifies the inter-market transmission strength through the partial derivative of volatility with respect to returns and the liquidity difference decay function, and generates the risk contagion strength factor.

[0175] The contagion intensity matrix unit is used to perform three-dimensional matrix processing on the risk contagion intensity factors and construct a risk contagion intensity matrix that includes the transmission relationship between the stock, foreign exchange and commodity markets.

[0176] Preferably, the high-risk path blocking module 620 provided in this application is configured with the following units:

[0177] The high-risk edge screening unit is used to perform dynamic threshold screening on the risk contagion intensity matrix, extract cross-market transmission relationship edges that exceed the preset risk threshold, and generate a set of high-risk transmission edges.

[0178] The risk transmission graph construction unit is used to construct a directed graph structure for the set of high-risk transmission edges, connect stock market nodes, foreign exchange market nodes and commodity market nodes to form a three-dimensional risk transmission path network, generate a directed risk transmission graph, and calculate the sum of the in-edge contagion strength of each node to generate the node in-degree risk value.

[0179] The original weight adjustment generation unit is used to process the current asset weights and market volatility characteristics obtained from the risk transmission directed graph through the strategy network, and combine real-time return and volatility data to map the weight adjustment scheme and generate the original asset weight adjustment amount.

[0180] The risk isolation constraint unit is used to perform risk isolation constraint processing on the original asset weight adjustment amount. Based on the node in-degree risk value, the increase of high-risk asset weight is compressed proportionally to generate the constrained adjustment amount.

[0181] The capital conservation normalization unit is used to perform capital conservation normalization on the constrained adjustment amount, scale the adjustment amount of each asset to keep the total weight at 1, and generate the asset weight adjustment amount.

[0182] Preferably, the isolation effect quantification module 630 provided in this application is configured with the following units:

[0183] The combined weight update unit is used to perform combined weight update processing on the asset weight adjustment amount, and add the current holding weight to generate the adjusted ratio, thus generating the adjusted weight vector.

[0184] The risk exposure calculation unit is used to perform risk exposure calculation on the adjusted weight vector, perform the product operation of the corresponding node weights on the risk transmission edge to quantify cross-market exposure, and generate risk exposure penalty terms;

[0185] The matrix trace operation processing unit is used to perform matrix trace operation on the risk contagion intensity matrix and asset weight adjustment amount, construct the adjustment amount diagonal matrix and perform trace operation with the contagion intensity matrix, and synthesize the isolation effect reward based on the risk exposure penalty term.

[0186] Preferably, the strategy countermeasure optimization module 640 provided in this application is configured with the following units:

[0187] The extreme event feature extraction unit is used to extract features from a preset historical extreme event dataset, extract market volatility mutation features and liquidity depletion feature patterns, and generate black swan event feature vectors.

[0188] The adversarial perturbation generation unit is used to generate adversarial perturbations from the risk contagion intensity matrix and the black swan event feature vector. It learns the distribution pattern of extreme events and constructs matrix offsets through generative adversarial networks to generate adversarial sample matrices.

[0189] The strategy difference minimization unit is used to minimize the strategy difference based on the adversarial sample matrix and the risk contagion intensity matrix. It calculates the Euclidean distance gradient of the output vector of the strategy network in the real environment and the adversarial environment, generates the strategy network parameter update amount, and updates the strategy network parameters.

[0190] Preferably, the dynamic blocking decision execution module 650 provided in this application is configured with the following units:

[0191] The decision model loading unit is used to load decision models onto policy network parameters, initialize the reinforcement learning policy network architecture and configure weight parameters, and generate a real-time decision engine.

[0192] The market feature vectorization unit is used to perform feature vectorization processing on real-time cross-market market data obtained from the financial market data interface, and integrate volatility, return and liquidity indicators of stock, foreign exchange and commodity markets to generate input vectors for the strategy network.

[0193] The portfolio adjustment instruction generation unit is used to process the input vector of the strategy network to generate weight adjustment amounts. It calculates the weight changes of each asset through forward propagation of the real-time decision engine and generates executable portfolio adjustment instructions.

[0194] In one embodiment, this application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described reinforcement learning-based dynamic investment decision-making method.

[0195] In one embodiment, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described reinforcement learning-based dynamic investment decision-making method.

[0196] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0197] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0198] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A dynamic investment decision-making method based on reinforcement learning, characterized in that, The method comprises the following steps: S1: modeling the dynamic infection intensity of real-time cross-market market data obtained through a financial market data interface, calculating the cross-market volatility sensitivity, liquidity difference attenuation term and Granger causality identification, and generating a risk infection intensity matrix; S2: blocking high-risk paths of the risk infection intensity matrix, identifying key transmission paths based on a directed graph and imposing node in-degree risk constraints to generate asset weight adjustment amounts; S3: quantifying the isolation effect of the asset weight adjustment amounts and the risk infection intensity matrix, calculating the product penalty term of the adjusted asset weight on the risk transmission edge and the trace operation gain term of the risk infection intensity matrix and the adjustment amount matrix, and weighted summing to generate an isolation effect reward; S4: strategy countermeasures optimization of the risk infection intensity matrix and the isolation effect reward, combining historical extreme event features obtained from a preset historical extreme event data set to generate disturbance samples and optimizing through minimizing the gradient difference between real decisions and countermeasures decisions to generate strategy network parameters; S5: dynamic blocking decision execution based on the strategy network parameters, processing real-time cross-market market data through a strategy network to generate asset weight adjustment amounts, and generating portfolio adjustment instructions, which are used to indicate weight redistribution of an asset portfolio.

2. The method of claim 1, wherein, The S1 comprises: S11: dynamic conditional correlation analysis and processing of real-time cross-market market data obtained through a financial market data interface, calculating the volatility rate covariance of stock, foreign exchange and commodity markets to generate a correlation coefficient matrix; S12: infection intensity factor calculation and processing of the correlation coefficient matrix, quantifying the transmission intensity between markets through the partial derivative of volatility rate to yield rate and the liquidity difference attenuation function to generate a risk infection intensity factor; S13: three-dimensional matrix processing of the risk infection intensity factor, constructing a risk infection intensity matrix containing the transmission relationship between stock-foreign exchange-commodity markets.

3. The method of claim 1, wherein, The S2 comprises: S21: dynamic threshold screening processing of the risk infection intensity matrix, extracting cross-market transmission relationship edges that exceed a preset risk threshold to generate a high-risk transmission edge set; S22: directed graph structure construction processing of the high-risk transmission edge set, connecting stock market nodes, foreign exchange market nodes and commodity market nodes to form a three-dimensional risk transmission path network, generating a risk transmission directed graph, and calculating the sum of node in-edge infection intensities to generate a node in-degree risk value; S23: strategy network processing of the current asset weight and market volatility characteristics obtained from the risk transmission directed graph, combining real-time yield rate and volatility rate data to map weight adjustment schemes, and generating original asset weight adjustment amounts; S24: risk isolation constraint processing of the original asset weight adjustment amounts, proportionally compressing the high-risk asset weight increase according to the node in-degree risk value to generate constrained adjustment amounts; S25: fund conservation normalization processing of the constrained adjustment amounts, scaling each asset adjustment amount to keep the total weight at 1 to generate asset weight adjustment amounts.

4. The method of claim 1, wherein, The S3 comprises: S31: Perform a portfolio weight adjustment amount combination weight update process, superimpose the current position weight to generate an adjusted proportion, and generate an adjusted weight vector; S32: Perform a risk exposure calculation process on the adjusted weight vector, perform a corresponding node weight product operation on the risk transmission edge to quantify the cross-market exposure, and generate a risk exposure penalty term; S33: Perform a matrix trace operation process on the risk contagion intensity matrix and the asset weight adjustment amount, construct an adjustment amount diagonal matrix and perform a trace operation with the contagion intensity matrix, and weight and synthesize the isolation effect reward based on the risk exposure penalty term.

5. The method of claim 4, wherein, The calculation formula of the isolation effect reward is: wherein, is an isolation effect reward, is a risk exposure penalty weight coefficient, is a risk transmission edge set, is a risk contagion intensity from market i to j, are the adjusted asset weights of market i and j, respectively, is an isolation gain weight coefficient, denotes a matrix trace operation, is a risk contagion intensity matrix, is a diagonal matrix with asset weight adjustment quantities as diagonal elements.

6. The method of claim 1, wherein, The S4 includes: S41: Extract features from a preset historical extreme event data set, extract market volatility rate mutation features and liquidity exhaustion feature patterns, and generate a black swan event feature vector; S42: Perform adversarial noise generation on the risk contagion intensity matrix and the black swan event feature vector, learn the extreme event distribution rule through a generative adversarial network, construct a matrix offset, and generate an adversarial sample matrix; S43: Perform strategy difference minimization processing based on the adversarial sample matrix and the risk contagion intensity matrix, calculate the Euclidean distance gradient of the output vector of the strategy network in the real environment and the adversarial environment, generate a strategy network parameter update amount, and update the strategy network parameters.

7. The method according to any one of claims 1 to 6, characterized in that, The S5 includes: S51: Perform a decision model loading process on the strategy network parameters, initialize a reinforcement learning strategy network architecture and configure weight parameters, and generate a real-time decision engine; S52: Perform feature vectorization processing on real-time cross-market market data obtained through a financial market data interface, integrate volatility, yield, and liquidity indicators of stock, foreign exchange, and commodity markets, and generate a strategy network input vector; S53: Perform a weight adjustment amount generation process on the strategy network input vector, calculate the weight change value of each asset through the forward propagation of the real-time decision engine, and generate executable portfolio adjustment instructions.

8. A dynamic investment decision system based on reinforcement learning, characterized in that, The system includes: A risk contagion intensity modeling module for dynamically modeling the real-time cross-market market data obtained through a financial market data interface, calculating cross-market volatility sensitivity, liquidity difference decay terms, and Granger causality indicators, and generating a risk contagion intensity matrix; A high-risk path blocking module for blocking high-risk paths based on the risk contagion intensity matrix, identifying key transmission paths based on a directed graph, and applying node in-degree risk constraints to generate asset weight adjustment amounts; An isolation effect quantification module for quantifying the isolation effect of the asset weight adjustment amount and the risk contagion intensity matrix, calculating the product penalty term of the adjusted asset weight on the risk transmission edge and the trace operation gain term of the risk contagion intensity matrix and the adjustment amount matrix, and weighting and summing to generate an isolation effect reward; A strategy adversarial optimization module for performing strategy adversarial optimization on the risk contagion intensity matrix and the isolation effect reward, generating a disturbance sample based on historical extreme event features obtained from a preset historical extreme event data set, and optimizing through gradient minimization of the difference between real decisions and adversarial decisions to generate strategy network parameters; A dynamic blocking decision execution module is configured to perform a dynamic blocking decision based on the policy network parameters, generate an asset weight adjustment amount by processing real-time cross-market market data through a policy network, and generate a portfolio adjustment instruction for indicating a weight redistribution of an asset portfolio. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The computer program, when executed by the processor, implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the method of any one of claims 1 to 7.

Images