Transaction strategy generation method and system based on graph neural network and reinforcement learning

Abstract

This invention provides a method and system for generating trading strategies based on graph neural networks and reinforcement learning, belonging to the field of computer data processing. It aims to solve the problem that existing technologies cannot effectively represent high-order correlations in multimodal data, leading to high training resource consumption and poor robustness in reinforcement learning models. The method includes: performing tensor decomposition on multimodal data composed of trading, social, and user behavior data to obtain latent feature representations; constructing a heterogeneous graph based on these latent feature representations and generating node embedding vectors using a graph neural network; fusing these node embedding vectors with market trend feature vectors to form the state vector of the reinforcement learning model; and inputting this state vector into a reinforcement learning strategy network to generate trading actions in a continuous action space. This invention improves data processing efficiency, accelerates model convergence, reduces resource consumption, and enhances model performance and robustness by constructing an information-complete state space.

Description

Technical Field

[0001] This invention relates to the fields of computer data processing and artificial intelligence, specifically to a method and system for generating trading strategies based on graph neural networks and reinforcement learning. More particularly, it relates to a method and system for processing multimodal heterogeneous data and optimizing computer decision-making models. Background Technology

[0002] In applications that leverage computer models to process complex data for sequential decision-making, such as quantitative financial analysis, a core technical challenge is how to efficiently process and represent input data. Traditional methods typically rely on time-series data, such as prices and trading volumes, to build statistical models for decision-making. However, these methods suffer from bottlenecks in processing multi-source, heterogeneous data. They often process different modalities, such as time-series and relational data, separately, leading to inefficient data representation, loss of key information, and difficulty in capturing the deep, complex relationships behind the data, such as implicit relationships between data nodes and the propagation paths of information within the network.

[0003] To address these issues, some existing technologies attempt to integrate multi-source data, employ graph neural networks for information updates, and then combine deep reinforcement learning to dynamically adjust decisions. However, when constructing the graph structure, these methods typically use flattened feature vectors as state inputs. This approach fails to effectively represent the topological information inherent in the original data, nor does it effectively handle and characterize the high-order, multi-dimensional potential relationships hidden in multimodal data. This results in low efficiency in representing the state space of the reinforcement learning model, high information redundancy, requiring longer training times and more computational resources for convergence, and ultimately leading to a model that is sensitive to data noise and lacks robustness.

[0004] Therefore, existing technologies urgently need to solve a technical problem: how to design an efficient data processing and representation method that can extract and fuse high-order structural information and temporal dynamic information from multi-source heterogeneous raw data to construct a reinforcement learning state space with high information density and high computational efficiency, thereby reducing the resource consumption of model training and improving the performance and stability of the final model. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a trading strategy generation method and system based on graph neural networks and reinforcement learning. This aims to solve the technical problems of existing technologies in processing multimodal heterogeneous data containing information on transactions, social interactions, and behaviors. Specifically, it addresses the difficulty in constructing a unified data structure that effectively represents high-order and complex relationships between data, leading to high redundancy and missing structural information in the state vector inputs of subsequent machine learning models. This, in turn, results in high computational resource consumption, slow convergence speed, and poor robustness of the final generated model during model training.

[0006] A trading strategy generation method based on graph neural networks and reinforcement learning, provided by the present invention, includes: Obtain input data that includes transaction data, social data, user behavior data, and market data; Tensor decomposition is performed on the multimodal data consisting of transaction data, social data, and user behavior data to obtain latent feature representations; Extract market trend feature vectors from the market trend data; A heterogeneous graph is constructed based on the latent feature representation, and a graph neural network is used to process the heterogeneous graph to generate node embedding vectors. The node embedding vector is fused with the market trend feature vector to form the state vector of the reinforcement learning model; The state vector is input into a reinforcement learning policy network to generate trading actions in a continuous action space.

[0007] Preferably, the fusion method involves concatenating the node embedding vector with the market trend feature vector.

[0008] Preferably, the fusion method involves weighted fusion of the node embedding vector and the market trend feature vector through a linear transformation layer.

[0009] Preferably, the tensor decomposition operation is Tucker decomposition.

[0010] Preferably, the graph neural network is a graph attention network.

[0011] Preferably, the reinforcement learning policy network adopts an Actor-Critic architecture.

[0012] Preferably, the Actor-Critic architecture employs either a dual-delay deep deterministic policy gradient algorithm or the Actor-Critic algorithm.

[0013] Preferably, the heterogeneous graph includes user nodes, asset nodes, and behavior nodes.

[0014] Preferably, the transaction action is an action vector that includes components representing the buy ratio, sell ratio, and / or position adjustment magnitude.

[0015] A trading strategy generation system based on graph neural networks and reinforcement learning, according to the present invention, includes: The data processing module is used to acquire input data including transaction data, social data, user behavior data and market data, perform tensor decomposition on the multimodal data composed of the transaction data, social data and user behavior data to obtain latent feature representations, and extract market feature vectors from the market data. The graph embedding module is configured to receive the latent feature representation from the data processing module, construct a heterogeneous graph based on the latent feature representation, and process the heterogeneous graph using a graph neural network to generate node embedding vectors. A state construction module is configured to receive the node embedding vector from the graph embedding module and the market trend feature vector from the data processing module, and to fuse the node embedding vector and the market trend feature vector to form a state vector of the reinforcement learning model. A strategy generation module is configured to receive the state vector from the state construction module and input the state vector into a reinforcement learning policy network to generate trading actions in a continuous action space.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention transforms high-dimensional, sparse, and heterogeneous raw data into low-dimensional, dense structured embedding vectors through tensor decomposition and graph neural networks, thereby improving the processing efficiency and representation capability of multimodal heterogeneous data. It not only effectively compresses the data and reduces the computational burden of subsequent models, but also deeply mines the hidden high-order correlations in the data, providing the computer system with inputs that are more informative and have a higher signal-to-noise ratio than the original features.

[0017] 2. This invention constructs a state space with complete information by fusing graph embedding vectors with temporal feature vectors, providing the model with the inherent structural information of the data, reducing the complexity of the learning task, thereby accelerating the convergence speed of the model and improving the robustness and generalization ability of the final generated model.

[0018] 3. The pipeline processing method proposed in this invention, which goes from data dimensionality reduction and graph embedding to state fusion, enables the core reinforcement learning policy network to be a lighter structure by pre-extracting efficient feature representations, thereby reducing the computational resource consumption of sequential decision-making on complex data and the inference latency of single decision.

[0019] 4. The decision model generated by the method of this invention has significantly improved key performance evaluation indicators compared with the baseline model, proving that the method can improve the decision performance and stability of computer models. Attached Figure Description

[0020] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A schematic diagram of the overall process of a trading strategy generation method based on graph neural networks and reinforcement learning provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a system architecture for GNN and RL coupling provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of state space mapping provided for an embodiment of the present invention; Figure 4 This is a signaling interaction timing diagram provided for an embodiment of the present invention; Figure 5 This is a schematic diagram of the system implementation environment provided in an embodiment of the present invention. Detailed Implementation

[0021] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0022] Example 1 This embodiment provides an implementation scheme for a trading strategy generation method based on graph neural networks and reinforcement learning. For example... Figure 1 As shown, this method aims to construct a reinforcement learning state space with high information density and high computational efficiency by deeply fusing high-order structural information and market time-series dynamic information contained in multimodal data, thereby improving the efficiency and performance of computer decision-making models.

[0023] In a specific application scenario, this method can be executed by one or more computer servers equipped with necessary computing resources such as processors, memory, storage, and network interfaces. The method includes: Step S1: Acquire and process multimodal input data. Specifically, the computer system first acquires raw input data from multiple data sources. For example... Figure 5As shown, the data sources include, but are not limited to: exchange market data application programming interfaces (APIs) for providing real-time price and volume trading data; social media APIs for providing social data such as user discussions and following relationships; and internal databases for storing user behavior data such as historical trading records and comments. Accordingly, the input data acquired includes at least trading data, social data, user behavior data, and market data.

[0024] Step S2: Perform structured processing on the data constituting the multimodal component of the input data to obtain latent feature representations. Step S2 includes: Step S2.1: Align and organize the three types of heterogeneous data—transaction data, social data, and user behavior data—to construct a unified high-order data structure, such as a third-order data tensor. The three dimensions of this tensor can represent users, financial assets, and interaction behaviors, respectively. The value of an element (i, j, k) in the tensor can represent the event or its intensity where user i performs behavior k (e.g., buy, sell, post a positive comment, etc.) on asset j.

[0025] Step S2.2: Perform tensor decomposition on the high-order data tensor. Tensor decomposition is a method that approximates a high-dimensional tensor as a product of multiple low-dimensional matrices or vectors. Its core objective is to achieve dimensionality reduction and extraction of latent structural features while preserving key information. In a specific embodiment of this invention, the Tucker decomposition algorithm can be used, which decomposes the original third-order tensor into a product of a core tensor and three factor matrices. These three factor matrices represent the low-dimensional latent features in the three dimensions of user, asset, and behavior. Through this operation, not only is the original high-dimensional, sparse data compressed into a low-dimensional, dense representation, effectively reducing the complexity of subsequent calculations, but more importantly, it explicitly mines the implicit high-order interaction relationships between different modalities of data, obtaining latent feature representations that are more informative than the original features.

[0026] Simultaneously, the computer system also extracts a market characteristic vector M from the market data. This vector M typically consists of a series of time-series technical indicators related to the target asset. For example, it may include candlestick data consisting of opening prices, closing prices, highest prices, lowest prices, and trading volumes over the past N time units (e.g., 60 minutes), as well as moving averages, relative strength index (RSI), Bollinger Bands, and other indicators calculated from this data. This market characteristic vector M is primarily used to characterize the market's macroeconomic dynamics and price trends.

[0027] Step S3: Based on the latent feature representation obtained through tensor decomposition, construct a heterogeneous graph and generate node embedding vectors. For example... Figure 2As shown, the heterogeneous graph is a graph structure containing multiple types of nodes and multiple types of edges. In a specific embodiment of the present invention, the heterogeneous graph may contain at least three types of nodes: user nodes, asset nodes, and behavior nodes. User nodes represent individuals participating in market transactions or discussions; asset nodes represent traded financial assets, such as stocks; and behavior nodes represent various operations that users may perform. The edges between nodes are defined based on their latent feature relationships. For example, if two users have high similarity in their representation vectors in the latent feature space, an edge can be established between their corresponding user nodes to represent that they may have similar investment preferences or behavioral patterns.

[0028] After constructing the heterogeneous graph, a graph neural network model is used to process the graph to generate embedding vectors for each node. A graph neural network is a deep learning model specifically designed for processing graph-structured data. It learns the vector representation of each node by iteratively aggregating information from neighboring nodes on the graph. In a preferred embodiment of this invention, the graph neural network used is a graph attention network (GAT). The core advantage of a graph attention network is its ability to assign different attention weights to each neighboring node of the central node through a self-attention mechanism. This means that during information aggregation, the model can adaptively determine which neighboring nodes' information is more important, which is particularly effective for processing heterogeneous graphs with diverse node and edge types.

[0029] The processing of a graph attention network is roughly as follows: For each node in the graph, the model examines all its neighboring nodes and calculates an attention score based on the features of the neighboring nodes and the center node. Then, the normalized attention score is used to perform a weighted summation of the features of the neighboring nodes to obtain the updated representation of the center node. This process can be repeated multiple times, enabling nodes to capture information from multi-hop neighbors. After processing by the graph attention network, each node in the graph (especially the asset nodes or user nodes we are interested in) receives a low-dimensional, dense vector representation, namely the node embedding vector Z. This node embedding vector Z encodes the node's own attribute information and its topological structure information in the entire complex relationship network, and can be regarded as a highly condensed summary of the node's role.

[0030] Step S4: To construct a complete decision-making basis, features need to be fused to build a state vector for reinforcement learning. For example... Figure 3 As shown, this step fuses the node embedding vector Z generated in step S3 with the market trend feature vector M extracted in step S1, so that the subsequent decision model can utilize both structural and temporal information simultaneously. In this embodiment, a direct and effective fusion method is to concatenate the node embedding vector Z with the market trend feature vector M. Specifically, if Z is d... z A d-dimensional vector, M being a d-dimensional vector.m If the vectors are dimensional, then concatenating them will form a d-dimensional vector. z + d m A vector of dimension S, which is the state vector S of the reinforcement learning model at the current decision moment.

[0031] Understandably, the advantage of constructing the state vector S in this way lies in its information completeness. It not only includes dynamic time-series information such as market prices and trading volumes provided by the market feature vector M, but also deep structural information mined from user social relationships and trading behaviors provided by the node embedding vector Z. Compared to using only time-series features or the original high-dimensional features, this fused state vector S has a higher signal-to-noise ratio and information density, providing a more comprehensive and easier-to-learn decision basis for the reinforcement learning model. This helps reduce the learning difficulty of the model, improve training efficiency, and ultimately enhance performance.

[0032] Step S5: Using the constructed state vector, generate continuous trading actions through a reinforcement learning model. The fused state vector S is used as input to the reinforcement learning policy network to generate the final trading actions. In this embodiment, the reinforcement learning model adopts an Actor-Critic architecture. This architecture is a mainstream framework in the field of reinforcement learning, containing a policy network (i.e., the Actor) and a value network (i.e., the Critic), which work together to learn the optimal policy.

[0033] Specifically, the Actor network is responsible for directly outputting a specific action based on the input state vector S. In this invention, to achieve refined money management, the action space is set to be continuous. For example, the output of the Actor network can be one or more continuous values ​​within a specific range, such as a value in the interval [-1, 1], which can be interpreted as a target position ratio (e.g., 0.5 represents using 50% of the funds to go long, and -0.2 represents using 20% ​​of the funds to go short). As an optional implementation, the trading action can be an action vector containing multiple components, which respectively represent the proportion of currently available funds to buy, the proportion of currently held positions to sell, or directly adjust the position of an asset to a value of a certain target range. This setting of continuous actions, compared to discrete buy / sell / hold actions, avoids the information loss caused by action quantification, making strategy adjustments smoother and more flexible.

[0034] Correspondingly, the Critic network is responsible for evaluating the merits of the action A chosen by the Actor network in a given state S. It typically outputs a Q value, which represents the expected total reward that may be obtained in the future after performing action A in state S.

[0035] like Figure 4As shown, during the training process, the agent (i.e., the reinforcement learning model) first obtains the current state S from the environment (i.e., the financial market) at each decision time t. t Then, the Actor network inside the smart body, based on S... t Calculate and output an action A t Action A t After execution, the environment will transition to the next state S. {t+1} and provide a reward signal R. t Reward R t The calculation method can vary; for example, it can be the change in portfolio value within that time step. The Critic network utilizes (S... t A t , R t , S {t+1} The Actor network updates its state-action value estimate using empirical data, striving to make the estimate closer to the true future reward. The Critic network, on the other hand, adjusts its parameters based on guidance from the Critic network (usually the gradient of the Q-value), making it more inclined to output actions that yield higher Q-values. Through numerous such exploration-exploitation cycles, the Actor and Critic networks iteratively optimize, ultimately enabling the Actor network to learn a strategy that makes optimal decisions in a given state.

[0036] During the actual decision-making (or reasoning) phase, the system uses only the trained Actor network. At each decision point, the system executes the aforementioned process to construct the latest state vector S, which is then input into the Actor network. The network immediately outputs a continuous trading action, which is subsequently sent to the trading execution system to place the order.

[0037] This embodiment successfully integrates high-order correlation information from multimodal data into a reinforcement learning decision-making framework. Backtesting experiments show that the decision model generated using this method improves key performance indicators compared to the baseline model.

[0038] Example 2 This embodiment is an optional implementation of Embodiment 1. Its core difference lies in the use of a more advanced algorithm in the reinforcement learning policy generation step, aiming to further improve the stability of model training and the performance of the final policy. Specifically, this embodiment replaces the basic Actor-Critic architecture with a dual-delay deep deterministic policy gradient algorithm.

[0039] In this embodiment, the steps of data acquisition and processing, heterogeneous graph construction and graph embedding generation, as well as feature fusion and state vector construction, are all the same as in Embodiment 1. That is, the system also generates a state vector S that integrates structural and temporal information for each decision moment through tensor decomposition, graph attention network processing, and feature concatenation.

[0040] The improvements in this embodiment focus on the policy generation and training stages. It should be noted that the standard Actor-Critic algorithm, when dealing with continuous action space problems, may overestimate the Q-value; that is, the Critic network tends to give overly optimistic value assessments, which can mislead the Actor network's updates and lead to unstable policy learning. The TD3 algorithm is designed to solve this technical problem.

[0041] Specifically, the TD3 algorithm introduces several key technological improvements based on the standard Actor-Critic architecture. First, it employs a truncated double-Q learning mechanism, simultaneously training a pair of Critic networks and taking the smaller of the two target Q values ​​to effectively suppress overestimation of value. Second, it implements delayed policy updates, meaning the Actor network updates less frequently than the Critic network, ensuring policy updates are based on more accurate value estimates and increasing the stability of the learning process. Third, it introduces target policy smoothing, adding a small amount of truncated random noise to the target actions to smooth the value function. This allows the policy to produce similar actions for similar states, reducing the policy's sensitivity to small errors in the value function and making the learned policy more robust.

[0042] Therefore, during the training process in this embodiment, the system maintains one Actor network, two Critic networks, and their respective target networks. In each training iteration, the Critic network is updated using a double Q-learning mechanism, while the Actor network is updated at a lower frequency using smoothed target and smoothed policy gradients.

[0043] By employing the TD3 algorithm, this embodiment exhibits a more stable model training process and better convergence compared to Embodiment 1. The trading strategy represented by the final trained Actor network demonstrates stronger robustness in the face of market noise and unseen data, and its decision sequence is typically smoother, avoiding overly frequent or drastic position adjustments due to model instability. This further enhances the reliability and profitability stability of the strategy in practical applications.

[0044] Example 3 This embodiment is another optional implementation of Embodiment 1. The core difference lies in the construction method of the state vector, namely the specific implementation of the feature fusion and state vector construction steps. Embodiment 1 adopts a simple feature concatenation method, while this embodiment explores a more adaptive weighted fusion method.

[0045] The steps in this embodiment, such as data acquisition and processing, heterogeneous graph construction and graph embedding generation, and reinforcement learning policy generation, can be consistent with those in Embodiment 1. For example, Tucker decomposition is still used to extract latent features, a graph attention network is used to generate node embedding vectors Z, and a basic Actor-Critic architecture is used for policy learning.

[0046] like Figure 3 As shown in the figure, two fusion methods are illustrated: concatenation and linear transformation. In this embodiment, instead of simply concatenating the node embedding vector Z and the market trend feature vector M along the feature dimension, a learnable linear transformation layer is used to perform weighted fusion on them.

[0047] As a specific implementation, the node embedding vector Z and the market trend feature vector M can be concatenated to obtain a temporary long vector [Z, M]. This long vector is then input into a fully connected neural network layer. This fully connected layer can be a simple linear layer, or a single-layer or multi-layer perceptron containing a non-linear activation function (such as ReLU). The output of this layer is the final reinforcement learning state vector S. This process can be represented as: S = g(W * [Z, M] + b) Here, W and b are the weight matrix and bias vector of the fully connected layer, respectively, and g is an optional non-linear activation function. Crucially, the weights W and bias b are not fixed but are learned and optimized along with the entire reinforcement learning model through backpropagation during end-to-end training.

[0048] The advantage of this fusion approach lies in its adaptability. Simple concatenation implicitly assumes that information from the graph structure and information from the time-series market have equal or fixed importance for the final decision. However, by introducing a learnable linear transformation layer, the model is endowed with the ability to automatically learn how to optimally combine these two information sources. During training, the model can autonomously adjust the weights W based on the final reward signal, thus deciding whether to focus more on the graph embedding vector Z (i.e., structural information) or on the market feature vector M (i.e., time-series information), and can even learn the complex nonlinear interactions between the two.

[0049] For example, during periods of strong market trends, the model may learn to assign higher weights to the market feature vector M; while during periods of market volatility, lack of clear trends, but stronger correlation between social network sentiment or large investor behavior, the model may learn to assign higher weights to the node embedding vector Z.

[0050] Therefore, this embodiment provides a more flexible and effective feature fusion mechanism. For certain specific datasets or market environments, this adaptive weighted fusion method may be more effective than simple concatenation in extracting the most critical combined features for decision-making, thereby potentially achieving better model performance and stronger environmental adaptability.

[0051] Example 4 This embodiment describes, from a system implementation perspective, how to deploy the aforementioned method as a runnable and scalable financial trading strategy generation system. For example... Figure 5 As shown, this diagram illustrates the system implementation environment provided by an embodiment of the present invention. The system modularizes the entire method flow, clearly defining the functions of each component and the data flow between them.

[0052] This financial trading strategy generation system logically comprises multiple modules that work together to form a complete workflow from data input to decision output. These modules mainly include: a data processing module, a graph embedding module, a state construction module, and a strategy generation module.

[0053] The data processing module, serving as the system's entry point, is primarily responsible for data acquisition and preliminary processing. This module is configured to connect to various upstream data sources, such as calling exchange market data APIs via network interfaces to obtain real-time trading data, calling social media APIs to obtain user comments and relationship data, and accessing internal databases to obtain historical behavioral data. After acquiring the data, this module performs the data preprocessing and feature extraction tasks described in Example 1, specifically including: constructing high-order tensors from multimodal trading, social, and user behavior data, and performing tensor decomposition operations on them to generate latent feature representations. Simultaneously, this module is also responsible for extracting structured market feature vectors M from the raw market data. The output of this module is sent to the graph embedding module and the state construction module, respectively.

[0054] The core function of the graph embedding module is to process graph-structured data and generate high-quality node representations. This module receives latent feature representations from the data processing module and dynamically constructs or updates heterogeneous graphs based on these features. Internally, the module integrates a pre-trained or online-updable graph neural network model, such as the graph attention network mentioned in Example 1. The module periodically runs the graph neural network model, aggregating information and learning representations for nodes in the graph to compute the latest node embedding vector Z. The output of this module, the node embedding vector Z, is sent to the state construction module.

[0055] The state construction module simultaneously receives node embedding vector Z from the graph embedding module and market feature vector M from the data processing module. Based on a preset fusion strategy, this module fuses these two vectors, which come from different sources and have different properties, into a unified reinforcement learning state vector S. The fusion strategy can be the direct concatenation described in Example 1, or the weighted fusion through a linear transformation layer described in Example 3. The output of this module, the information-complete state vector S, is passed to the policy generation module.

[0056] The strategy generation module, as the core of the system's decision-making, receives the state vector S from the state construction module. Within this module, a pre-trained reinforcement learning policy network (i.e., an Actor network, such as one trained using the Actor-Critic or TD3 algorithm) is deployed. At each decision time, upon receiving the latest state vector S, this module immediately performs a forward inference calculation, outputting a continuous trading action A in real time. t For example, a numerical value representing the target position or an action vector containing the buy / sell ratio.

[0057] The overall system workflow is as follows: The system runs continuously as a whole. The data processing module periodically or event-drivenly pulls new data from the data source and processes it. When new data causes changes in underlying features, the graph embedding module updates the heterogeneous graph and node embeddings. At each point in time when a trading decision needs to be made (e.g., every minute), the state building module and strategy generation module are triggered, sequentially completing the reasoning process from state building to action generation. The trading action instructions output by the strategy generation module are ultimately sent to the downstream trading execution system, which then transforms these instructions into actual buy and sell orders through, for example, a brokerage interface, and sends them to the financial market for execution.

[0058] This embodiment provides a clear and modular system architecture, enabling the proposed method to be engineered and deployed. This architecture offers excellent scalability and maintainability; for example, any module can be easily replaced or upgraded (e.g., changing the graph neural network model or reinforcement learning algorithm) without altering the entire system. This provides a feasible foundation for engineering this technical solution and applying it to online decision-making systems that process real-time data streams.

[0059] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0060] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A trading strategy generation method based on graph neural networks and reinforcement learning, characterized in that, include: Obtain input data that includes transaction data, social data, user behavior data, and market data; Tensor decomposition is performed on the multimodal data consisting of transaction data, social data, and user behavior data to obtain latent feature representations; Extract market trend feature vectors from the market trend data; A heterogeneous graph is constructed based on the latent feature representation, and a graph neural network is used to process the heterogeneous graph to generate node embedding vectors. The node embedding vector is fused with the market trend feature vector to form the state vector of the reinforcement learning model; The state vector is input into a reinforcement learning policy network to generate trading actions in a continuous action space.

2. The trading strategy generation method based on graph neural networks and reinforcement learning according to claim 1, characterized in that, The fusion method involves concatenating the node embedding vector with the market trend feature vector.

3. The trading strategy generation method based on graph neural networks and reinforcement learning according to claim 1, characterized in that, The fusion method involves weighting and fusing the node embedding vector and the market trend feature vector through a linear transformation layer.

4. The trading strategy generation method based on graph neural networks and reinforcement learning according to any one of claims 1 to 3, characterized in that, The tensor decomposition operation is Tucker decomposition.

5. The trading strategy generation method based on graph neural networks and reinforcement learning according to any one of claims 1 to 3, characterized in that, The graph neural network is a graph attention network.

6. The trading strategy generation method based on graph neural networks and reinforcement learning according to any one of claims 1 to 3, characterized in that, The reinforcement learning policy network adopts an Actor-Critic architecture.

7. The trading strategy generation method based on graph neural networks and reinforcement learning according to claim 6, characterized in that, The Actor-Critic architecture employs either the dual-delay deep deterministic policy gradient algorithm or the Actor-Critic algorithm.

8. The trading strategy generation method based on graph neural networks and reinforcement learning according to any one of claims 1 to 3, characterized in that, The heterogeneous graph includes user nodes, asset nodes, and behavior nodes.

9. The trading strategy generation method based on graph neural networks and reinforcement learning according to any one of claims 1 to 3, characterized in that, The trading action is a motion vector that includes components representing the proportion of buying, the proportion of selling, and / or the magnitude of position adjustment.

10. A trading strategy generation system based on graph neural networks and reinforcement learning, characterized in that, include: The data processing module is used to acquire input data including transaction data, social data, user behavior data and market data, perform tensor decomposition on the multimodal data composed of the transaction data, social data and user behavior data to obtain latent feature representations, and extract market feature vectors from the market data. The graph embedding module is configured to receive the latent feature representation from the data processing module, construct a heterogeneous graph based on the latent feature representation, and process the heterogeneous graph using a graph neural network to generate node embedding vectors. A state construction module is configured to receive the node embedding vector from the graph embedding module and the market trend feature vector from the data processing module, and to fuse the node embedding vector and the market trend feature vector to form a state vector of the reinforcement learning model. A strategy generation module is configured to receive the state vector from the state construction module and input the state vector into a reinforcement learning policy network to generate trading actions in a continuous action space.

Images