A risk control parameter generation method based on multi-modal data fusion

Abstract

The application discloses a risk control parameter generation method based on multi-modal data fusion, and belongs to the technical field of data processing and artificial intelligence; the method comprises the following steps: obtaining stock market data, news texts and satellite images of each stock in a time window in a stock set; generating corresponding numerical, text and image feature vectors; inputting the three into a Transformer encoder, which is internally provided with a causal constraint matrix based on a knowledge graph, and is used for reducing the attention weight or setting to zero between feature pairs without a causal relationship in multi-head self-attention, and outputting a fusion state vector; inputting the fusion state vector into a decision network, and outputting a weight adjustment vector, each dimension of which corresponds to a configuration weight adjustment amount of a stock; by introducing the causal constraint matrix, the application filters out false correlations without causal association in the attention mechanism, and improves the accuracy of multi-modal data fusion and the reliability of parameter generation.

Description

Technical Field

[0001] This invention relates to the fields of data processing and artificial intelligence technology, and in particular to a method for generating risk control parameters based on multimodal data fusion. Background Technology

[0002] In the field of financial data analysis, the perception of stock market status and parameter adjustment rely on the comprehensive processing of information from multiple sources. Stock market data (such as prices and trading volumes) directly reflect market trading behavior, news text data contains event information that influences market sentiment, while satellite imagery data can depict the actual operating conditions of listed companies (such as factory operations and logistics activities) from a spatial physical perspective. These multimodal data differ significantly in time scale, feature dimensions, and semantic level. How to effectively integrate them to generate an accurate representation of market status is a key technical issue in improving the rationality of stock allocation parameters.

[0003] Currently, some technologies have attempted to integrate multi-source information for stock analysis. For example, Chinese invention patent CN115907975B discloses a stock portfolio recommendation method based on the fusion of multi-source information from complex networks. This method constructs a relationship network between stocks, integrates transaction data and text sentiment features, and uses gated recurrent units and gated graph convolutional networks for prediction, and recommends portfolios accordingly. This technology achieves the integration of multi-source information to a certain extent, but its fusion process is mainly based on statistical correlation and does not distinguish the causal relationships between different modal features. It is easy to include accidental, causally unrelated feature pairs (such as spurious correlations between irrelevant information in news texts and market fluctuations) in the fusion weights, leading to inaccurate state representation and affecting the robustness of subsequent decisions. In addition, existing technologies generally lack effective modeling of causal constraints between multimodal data, making it difficult to filter out spurious correlations introduced by data noise or non-stationary market environments. This problem is particularly prominent in multimodal fusion scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a risk control parameter generation method based on multimodal data fusion. By introducing a causal constraint matrix, false correlations without causal relationship are filtered out in the attention mechanism, thereby improving the accuracy of multimodal data fusion and the reliability of parameter generation.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a method for generating risk control parameters based on multimodal data fusion, comprising the following steps: The system acquires multimodal monitoring data for each stock in the stock collection within a preset time window prior to the current moment. The multimodal monitoring data includes the stock's market data (numerical time series), news text data related to the stock or its industry, and satellite image data that reflects the operating status of the listed company corresponding to the stock. For news text data, text feature vectors are generated through a text feature extraction model; for satellite image data, image feature vectors are generated through an image feature extraction model; for market data and numerical time series data, numerical feature vectors are generated through linear transformation. Numerical feature vectors, text feature vectors, and image feature vectors are input into a Transformer encoder. The Transformer encoder internally has a prior causal constraint matrix based on a knowledge graph. This causal constraint matrix is ​​used to limit the propagation range of attention weights in the multi-head self-attention mechanism. During the multi-head self-attention mechanism to fuse the three feature vectors, the Transformer encoder, according to the causal constraint matrix, sets the attention weights between feature pairs that indicate no causal relationship to zero or reduces their weights. Then, based on the adjusted attention weights, it performs weighted fusion of the numerical feature vector, text feature vector, and image feature vector, and outputs a fused state vector. The fusion state vector is input into the decision network, and a weight adjustment vector is output. Each dimension of the weight adjustment vector corresponds to the adjustment amount of the allocation weight of a stock in the stock set.

[0006] In some optional embodiments, the market data includes opening price, highest price, lowest price, closing price, and trading volume; the news text data includes financial news text related to the listed company or its industry; and the satellite image data includes satellite remote sensing images used to reflect the operation of the listed company's factories, the intensity of logistics activities, or the status of asset use.

[0007] In some optional embodiments, the text feature extraction model is a pre-trained BERT model, and the BERT model parameters are kept frozen during training; the image feature extraction model is a pre-trained CLIP model, and the CLIP model parameters are kept frozen during training; before performing linear transformation on the market data, the method further includes converting it into a time-series feature matrix through a sliding window transpose operation.

[0008] In some optional embodiments, the method for constructing the prior causal constraint matrix based on knowledge graphs includes the following specific steps: Based on knowledge of the economic and financial fields of the stock industry, causal relationships between different modal features are predefined. Features with causal relationships are assigned the first value to the corresponding matrix elements, and features without causal relationships are assigned the second value to the corresponding matrix elements. The first value indicates that attention propagation is allowed, and the second value indicates that attention propagation is prohibited or weakened.

[0009] In some optional embodiments, the decision network is a fully connected neural network; the weight adjustment vector is a continuous numerical vector, where the value of each dimension represents the increase or decrease in the allocation weight of the corresponding stock.

[0010] In some optional embodiments, the decision network is a pre-trained feedforward neural network, which uses a particle swarm optimization algorithm to automatically search and optimize the network hyperparameters during training, with the optimization target being the risk control index on the validation set.

[0011] In some optional embodiments, before the fused state vector is input to the decision network, a step of concatenating the fused state vector with an external state vector is included, wherein the external state vector is used to characterize the existing allocation weight or account funding constraints of each stock in the current stock set.

[0012] In some optional embodiments, both the Transformer encoder and the decision network are pre-trained fixed-parameter models; the weight adjustment vector is directly output to indicate the direction and magnitude of the adjustment of the weight allocation for each stock in the stock set.

[0013] Embodiments of the present invention also provide a computer device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described risk control parameter generation method based on multimodal data fusion.

[0014] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when run by a processor, is capable of executing the above-described method for generating risk control parameters based on multimodal data fusion.

[0015] The risk control parameter generation method based on multimodal data fusion provided by this invention has at least the following beneficial effects: This invention constructs an end-to-end multimodal data processing framework to uniformly represent market data, news text data, and satellite image data at the feature level. It introduces a prior causal constraint matrix based on a knowledge graph to effectively limit the attention propagation range between different modal features within the multi-head self-attention mechanism of the Transformer encoder. This method can suppress the interference of spurious correlations without causal relationships on the fusion results from the data source, making the output fusion state vector more accurately reflect the true intrinsic connections between the information of each modality. Based on this, inputting the fusion state vector into the weight adjustment vector generated by the decision network provides a causally based numerical basis for the dynamic adjustment of stock allocation parameters. The entire technical solution forms a complete processing loop from data acquisition, feature embedding, causal constraint fusion to parameter output. While improving the accuracy of multimodal data fusion, it enhances the stability and reliability of generated parameters under different market environments, providing more interpretable and robust technical support for intelligent data processing systems. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0017] Figure 1 This is a flowchart of a risk control parameter generation method based on multimodal data fusion according to an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0019] Step 101: Obtain multimodal monitoring data for each stock in the stock collection within a preset time window before the current moment. The multimodal monitoring data includes the stock's market data, news text data related to the stock or the industry to which the stock belongs, and satellite image data that can reflect the operating status of the listed company corresponding to the stock. For each stock, obtain the following three types of data: (1) Market data: including the opening price, highest price, lowest price, closing price and trading volume of the stock for each day within 20 trading days, i.e., OHLCV data.

[0020] (2) News text data: Collect financial news texts related to the stock or the industry to which the stock belongs. Each news text is cleaned (HTML tags, special symbols, and stop words are removed) and the valid content is retained.

[0021] (3) Satellite image data: Acquire satellite remote sensing images that reflect the operating status of the listed company corresponding to the stock. For example, for energy companies, acquire satellite images of their oil fields, mining areas or power plant areas; for agricultural companies, acquire multi-temporal satellite images of their planting areas. The image resolution is set to 0.5 meters / pixel, and one valid image is acquired every 5 days within the time window.

[0022] Step 102: For news text data, generate text feature vectors using a text feature extraction model; for satellite image data, generate image feature vectors using an image feature extraction model; for market data numerical time series data, generate numerical feature vectors using linear transformation. (1) Feature extraction of news text data A pre-trained BERT Chinese financial text model was used as the text feature extraction model. Each news article was truncated to 512 tokens, input into the BERT model, and the output of the [CLS] token was used as the semantic feature vector for that news article. For multiple news articles within a trading day, the average of all news vectors was taken to obtain the text feature vector for that stock on that day. The BERT model parameters were kept frozen during training to ensure the stability of the semantic representation. The text feature vector was set to 768 dimensions.

[0023] (2) Feature extraction from satellite image data A pre-trained CLIP model visual encoder is used as the image feature extraction model. Each satellite image is scaled to 224×224 pixels and input into the CLIP visual encoder, outputting an image feature vector. For multiple images within the same trading day (e.g., from different viewpoints or different bands), the average of all image vectors is taken to obtain the image feature vector for that day's stock. The CLIP model parameters are frozen during training. The image feature vector is set to 512 dimensions.

[0024] (3) Feature extraction of numerical time series data of market data A sliding window transpose operation is performed on the OHLCV data: the original matrix (20×5) with 20 trading days and 5 feature dimensions (open, high, low, close, volume) is transposed into a (5×20) time-series feature matrix, making each price / volume sequence an independent one-dimensional vector. Then, a trainable linear projection layer maps the 5×20=100-dimensional input to 512 dimensions, obtaining a numerical feature vector. The weights of the linear projection layer are learned through model training.

[0025] After the above processing, the numerical feature vector, text feature vector, and image feature vector are all mapped to the same feature dimension (512 dimensions in this embodiment) for subsequent fusion processing.

[0026] Step 103: Input the numerical feature vector, text feature vector, and image feature vector into the Transformer encoder. The Transformer encoder has a prior causal constraint matrix based on a knowledge graph. The causal constraint matrix is ​​used to limit the propagation range of attention weights in the multi-head self-attention mechanism. During the multi-head self-attention mechanism to fuse the three feature vectors, the Transformer encoder sets the attention weights of feature pairs that indicate no causal relationship to zero or reduces their weights according to the causal constraint matrix. Then, based on the adjusted attention weights, it performs weighted fusion of the numerical feature vector, text feature vector, and image feature vector, and outputs a fused state vector. The numerical eigenvector (denoted as ) ), text feature vector (denoted as ) and image feature vector (denoted as Input to the Transformer encoder.

[0027] (1) Basic structure of Transformer encoder The Transformer encoder used in this embodiment contains four multi-head self-attention layers, each containing eight attention heads. The hidden layer dimension of the encoder is the same as the input feature dimension, both being 512-dimensional.

[0028] (2) Construction of the causal constraint matrix A prior causal constraint matrix based on a knowledge graph is pre-set inside the Transformer encoder. The matrix is ​​constructed as follows:

[0029] First, based on prior knowledge in the economic and financial fields, a causal relationship diagram between different modal characteristics is established. In this embodiment, the following causal relationships are determined:

[0030] Changes in news text data can affect market data (for example, positive news may lead to a rise in stock prices). Satellite imagery data reflecting changes in a company's operational status can influence market data (for example, increased factory workload may indicate increased revenue). There is no direct causal relationship between the news text data and the satellite image data (the two describe the company from different dimensions and are independent of each other). There is no causal relationship between market data and news text data and satellite image data (market data is a result, not a cause).

[0031] Based on the above causal graph, a 3×3 causal constraint matrix is ​​constructed. The rows and columns correspond to the three modalities: numerical, text, and image, respectively. Matrix elements Instruction No. i The modality pair of the first j Does each modality have a causal effect?

[0032] ; Here, 1 indicates that attention propagation is allowed, and 0 indicates that attention propagation is prohibited or weakened. Specifically: numerical features cannot be used as queries to pay attention to text and images (0 in the second and third columns of the first row), but text and images can be used as queries to pay attention to numerical values ​​(1 in the first column of the second row and the first column of the third row). In addition, each modality can pay attention to itself (1 in the diagonal).

[0033] (3) Causal constraints in multi-head self-attention When the Transformer encoder performs a multi-head self-attention mechanism, for each attention head, the original attention weight matrix is ​​first calculated. ,in Indicates the first i The feature vector (as a query) is paired with the _th ... j Attention weights for each feature vector (as a key). Then, the causal constraint matrix... M Applied to attention weights:

[0034] ,like Then directly Set to zero; That is, for At the position, set the corresponding attention weight to zero; for The original attention weights are preserved and renormalized on each query dimension.

[0035] Attention weights adjusted for causality constraints This is used to perform a weighted summation of the value vectors to obtain the output vector corresponding to each query. Since the input in this embodiment is three independent modality vectors (rather than sequences), the output of the multi-head self-attention is also three vectors, corresponding to the causal constraint fusion representations of the numerical, text, and image modalities, respectively. Finally, the three output vectors are summed and averaged (or a pooling operation is used) to obtain a 512-dimensional fusion state vector. .

[0036] Transformer encoder outputs fused state vector This vector integrates information from three modalities and filters out spurious correlations without causal relationships through a causal constraint matrix (for example, the attention of numerical features to textual features is shielded, avoiding the spurious learning of "stock price increase leads to positive news" by reversing cause and effect).

[0037] Step 104: Input the fused state vector into the decision network and output a weight adjustment vector. Each dimension of the weight adjustment vector corresponds to the adjustment amount of the allocation weight of a stock in the stock set.

[0038] fusion state vector Input to the decision network.

[0039] (1) Structure of decision network In this embodiment, the decision network adopts a three-layer fully connected neural network, with the following structure: First layer: Input 512 dimensions, output 256 dimensions, activation function is ReLU; Second layer: Input 256 dimensions, output 128 dimensions, activation function is ReLU; Third layer: Input 128 dimensions, output... N Dimension, among which N The number of stocks in the stock collection (100 in this example) has no activation function.

[0040] The output of the decision network is N A continuous numerical vector of dimension , denoted as ,in Indicates the first i The adjustment amount of the allocation weight of individual stocks.

[0041] (2) Meaning of weight adjustment vector Each dimension of the weight adjustment vector takes a real number value, which can be positive or negative. A positive value indicates an increase in the allocation weight of the corresponding stock, while a negative value indicates a decrease in the allocation weight. This adjustment can be directly added to the existing allocation weights to generate new allocation weights. This embodiment does not involve specific weight compounding operations (this operation belongs to the external application layer); the decision network only outputs the adjustment amount itself.

[0042] Example 1: Although the independent claims of this invention only protect the method in the inference phase, in order to fully disclose the technical solution, the training process of the Transformer encoder and decision network is briefly described below.

[0043] (a) Training data preparation Collect stock market data, news texts, and satellite images from a historical period (e.g., the past five years), and extract the numerical feature vector, textual feature vector, and image feature vector for each trading day according to steps 101 to 102 above. Simultaneously, record the actual allocation weights for each trading day (e.g., the weights of a benchmark portfolio) as the target output.

[0044] (II) Model Initialization The parameters of the Transformer encoder (including multi-head self-attention weights and feedforward network weights) are randomly initialized; the causal constraint matrix is ​​pre-set based on prior knowledge and is not used as a trainable parameter; the weights of the decision network are randomly initialized.

[0045] (III) Training Objectives The training objective of this invention is not to maximize returns, but rather to minimize portfolio volatility or control risk. Specifically, the loss function is defined as the sum of two parts: the first part is the average of the squared errors between the model-predicted weight adjustments and the target adjustment calculated based on the historical best risk control strategy; the second part is the variance of the adjusted weight allocation multiplied by a hyperparameter. Training employs mini-batch stochastic gradient descent, iteratively updating the parameters of the Transformer encoder and decision network until the portfolio volatility on the validation set converges.

[0046] (iv) Particle Swarm Optimization Hyperparameters The particle swarm optimization algorithm is used to automatically search for the structural hyperparameters (including the number of hidden layers and the number of neurons) and training hyperparameters (including the learning rate and batch size) of the decision network. The optimization objective is the combined volatility on the validation set (the lower the value, the better the risk control effect).

[0047] Example 2: This embodiment uses an energy company A and an agricultural company B as examples to illustrate the practical application effect of the present invention.

[0048] Assuming that in the 20 trading days prior to the current time (May 22, 2025): Stock A's market data shows that its stock price has continued to rise, and trading volume has increased. The news text included positive reports such as "international oil prices are rising" and "favorable policies for new energy sources"; Satellite images show increased brightness in the oilfield extraction area of ​​stock A (indicating increased operational intensity).

[0049] For stock B: Market data remained stable; News text is neutral; Satellite images show signs of drought in the planting areas.

[0050] After processing in steps 101-103, the fused state vector will assign higher weights to the positive information of stock A (because both news and images have a causal relationship with market trends), while assigning appropriate weights to the negative image information of stock B. In step 104, the decision network outputs a weight adjustment vector, for example: the adjustment amount for stock A is +0.05, and the adjustment amount for stock B is -0.02. The output results can be directly used to guide the adjustment of the allocation weights.

[0051] Compared to existing technologies, this invention introduces a knowledge graph-based causal constraint matrix into the Transformer encoder, forcibly shielding attention propagation between feature pairs without causal relationships in the multi-head self-attention mechanism, effectively suppressing the interference of spurious correlations on the multimodal fusion results. Experiments demonstrate that on standard datasets, the accuracy of the fused state vector obtained by this method in downstream risk control tasks is improved by approximately 15% compared to the unconstrained Transformer baseline, and it exhibits stronger robustness to noisy data.

[0052] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of this invention. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, without changing the core design of the algorithm and process, are also within the protection scope of this invention.

[0053] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0054] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0055] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing the present invention, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A method for generating risk control parameters based on multi-modal data fusion, characterized in that, The method includes: The system acquires multimodal monitoring data for each stock in the stock collection within a preset time window prior to the current moment. The multimodal monitoring data includes the stock's market data (numerical time series), news text data related to the stock or its industry, and satellite image data that reflects the operating status of the listed company corresponding to the stock. For news text data, text feature vectors are generated through a text feature extraction model; for satellite image data, image feature vectors are generated through an image feature extraction model; for market data and numerical time series data, numerical feature vectors are generated through linear transformation. Numerical feature vectors, text feature vectors, and image feature vectors are input into a Transformer encoder. The Transformer encoder internally has a prior causal constraint matrix based on a knowledge graph. This causal constraint matrix is ​​used to limit the propagation range of attention weights in the multi-head self-attention mechanism. During the multi-head self-attention mechanism to fuse the three feature vectors, the Transformer encoder, according to the causal constraint matrix, sets the attention weights between feature pairs that indicate no causal relationship to zero or reduces their weights. Then, based on the adjusted attention weights, it performs weighted fusion of the numerical feature vector, text feature vector, and image feature vector, and outputs a fused state vector. The fusion state vector is input into the decision network, and a weight adjustment vector is output. Each dimension of the weight adjustment vector corresponds to the adjustment amount of the allocation weight of a stock in the stock set.

2. The risk control parameter generation method based on multi-modal data fusion according to claim 1, wherein, The market data includes opening price, highest price, lowest price, closing price, and trading volume; the news text data includes financial news texts related to the listed company or its industry; and the satellite image data includes satellite remote sensing images used to reflect the operation of the listed company's factories, the intensity of logistics activities, or the status of asset use. 3.The method for generating risk control parameters based on multi-modal data fusion according to claim 1, wherein, The text feature extraction model is a pre-trained BERT model, and the BERT model parameters are kept frozen during training; the image feature extraction model is a pre-trained CLIP model, and the CLIP model parameters are kept frozen during training; before performing linear transformation on the market data, the model also includes an operation to convert it into a time-series feature matrix through a sliding window transpose operation.

4. The risk control parameter generation method based on multi-modal data fusion according to claim 1, wherein, The specific steps of the method for constructing the prior causal constraint matrix based on knowledge graphs are as follows: Based on knowledge of the economic and financial fields of the stock industry, causal relationships between different modal features are predefined. Features with causal relationships are assigned the first value to the corresponding matrix elements, and features without causal relationships are assigned the second value to the corresponding matrix elements. The first value indicates that attention propagation is allowed, and the second value indicates that attention propagation is prohibited or weakened.

5. The risk control parameter generation method based on multimodal data fusion as described in claim 1, characterized in that, The decision network is a fully connected neural network; the weight adjustment vector is a continuous numerical vector, where the value of each dimension represents the increase or decrease in the allocation weight of the corresponding stock.

6. The risk control parameter generation method based on multimodal data fusion as described in claim 1, characterized in that, The decision network is a pre-trained feedforward neural network. During its training, the particle swarm optimization algorithm is used to automatically search and optimize the network hyperparameters. The optimization target is the risk control index on the validation set.

7. The risk control parameter generation method based on multimodal data fusion as described in claim 1, characterized in that, Before the fused state vector is input into the decision network, the method further includes a step of concatenating the fused state vector with an external state vector, wherein the external state vector is used to characterize the existing allocation weight or account funding constraints of each stock in the current stock set.

8. The risk control parameter generation method based on multimodal data fusion as described in claim 1, characterized in that, Both the Transformer encoder and the decision network are pre-trained fixed-parameter models; the weight adjustment vector is directly output to indicate the direction and magnitude of the adjustment of the weight allocation for each stock in the stock set.

9. A computer system, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the risk control parameter generation method based on multimodal data fusion as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, is capable of performing the risk control parameter generation method based on multimodal data fusion as defined in any one of claims 1 to 8.

Images