Method for constructing prediction model based on dynamic gating and cross-modal attention fusion

Abstract

The invention provides a construction method of a prediction model based on dynamic gating and cross-modal attention fusion, and belongs to the technical field of model construction. Comprising the following steps: constructing a feature coding layer, and respectively coding multi-source input data to obtain each modal feature vector; constructing a multi-source data interaction layer, and performing deep interaction on each modal feature vector; and finally, carrying out weighted fusion on the main modal features and the cross-modal interaction features based on a dynamic gating mechanism. Constructing a feature fusion layer, and performing time sequence pooling and full-connection fusion on the interacted multi-modal features to obtain a fusion feature vector; and constructing a quantile regression layer, and outputting a prediction result based on the fusion feature vector. According to the method, the prediction model is constructed by fusing the dynamic gating mechanism and the cross-modal attention mechanism, so that the problems of an existing prediction model in the aspects of deep fusion of multi-source heterogeneous data, cross-modal dynamic interaction and accurate quantification of tail risks are solved, and then efficient prediction of stock price collapse risks is realized.

Description

Technical Field

[0001] This invention relates to the field of model building technology, and in particular to a method for building a prediction model based on dynamic gating and cross-modal attention fusion. Background Technology

[0002] In the field of financial market research, predicting stock price crash risks is a crucial step in maintaining market stability and assisting investor decision-making. Accurately identifying stock price crash risks can provide effective early warnings for regulators, financial institutions, and investors, reducing losses caused by market volatility. However, current stock price crash risk prediction faces several challenges, including: From a data perspective, stock price-related information involves multi-source heterogeneous data, including fundamental data reflecting a company's operating conditions (such as financial statement indicators), technical indicators reflecting market trading dynamics (such as stock price trends and trading volume), and textual data influenced by policies and public opinion. These data exhibit significant modal differences and complex characteristics. Breaking down modal barriers and achieving deep integration and effective interaction of multi-source data is a key challenge in improving prediction accuracy. Traditional methods often process multi-source data separately or simply concatenate them, neglecting the potential correlations and complementary information between modalities, making it difficult for models to fully extract the value of the data.

[0003] At the model architecture level, existing predictive models lack sufficient granularity for handling multimodal data interactions. While self-attention mechanisms can capture feature dependencies within a single modality, cross-modal feature interactions often rely on simple concatenation or shallow fusion, lacking dynamic and adaptive fusion mechanisms. They cannot flexibly adjust the contribution of different modal features based on data characteristics. Furthermore, when facing tail risks in financial data (such as extreme market crashes), conventional predictive models often focus on mean or single quantile predictions, failing to comprehensively quantify risk uncertainty. Their sensitivity and ability to characterize tail risks are limited, failing to meet the demand for accurate risk warnings in practical applications.

[0004] Furthermore, financial data suffers from noise issues such as missing values ​​and outliers. Traditional preprocessing methods are not robust to data noise, and their adaptability to masking mechanisms and standardization is insufficient, easily leading to feature encoding bias and affecting model training performance. Moreover, in the feature fusion and prediction stages, the lack of efficient aggregation of time-series information and deep processing of multimodal fusion features makes it difficult to improve the model's generalization ability and prediction accuracy in complex financial scenarios.

[0005] Therefore, there is an urgent need for a stock price crash risk prediction model that can deeply integrate multi-source heterogeneous data, accurately capture intra-modal and inter-modal feature dependencies, effectively quantify risk uncertainty, and adapt to the characteristics of financial data. This model would address the shortcomings of existing technologies in multi-source data fusion, cross-modal interaction, and tail risk characterization, thereby improving the accuracy and reliability of stock price crash risk prediction. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a method for constructing a prediction model based on dynamic gating and cross-modal attention fusion. By fusing dynamic gating mechanism and cross-modal attention mechanism, a prediction model is constructed, which solves the problems of existing prediction models in deep fusion of multi-source heterogeneous data, cross-modal dynamic interaction and accurate quantification of tail risk, thereby achieving efficient prediction of stock price crash risk.

[0007] To achieve the above objectives, the present invention provides the following solution: A method for constructing a prediction model based on dynamic gating and cross-modal attention fusion includes the following steps: S1. Construct a feature encoding layer to encode the multi-source input data respectively to obtain the feature vectors of each modality. The encoding process includes linear transformation, GELU activation, layer normalization and Dropout regularization. S2. Construct a multi-source data interaction layer to perform deep interaction on the feature vectors of each modality. The deep interaction process includes strengthening the feature dependence within each modality feature vector through a self-attention mechanism, then realizing intermodal feature interaction through a cross-modal attention mechanism in an alternating master modality manner, and finally performing weighted fusion of master modality features and cross-modal interaction features based on a dynamic gating mechanism. S3. Construct a feature fusion layer to perform temporal pooling and fully connected fusion on the multimodal features after interaction to obtain a fused feature vector; S4. Construct a quantile regression layer and output the prediction result based on the fused feature vector.

[0008] Preferably, in S1, before encoding the multi-source input data respectively, the following steps are included: The multi-source input data is then suppressed for the effects of missing values ​​using a binary mask matrix. ; In the formula, A binary mask matrix is ​​used to mark the validity of the data, i.e. Indicates non-missing, where, Indicates an index. t Indicates time; Indicates missing, dimension is N × T , N For the sample size, T For time step; This indicates broadcasting the mask matrix to the feature dimension; ⊙ indicates element-wise multiplication; It represents the outer product.

[0009] Preferably, in S1, the multi-source input data is encoded respectively, including: The feature encoding layer maps the processed multi-source input data to a high-dimensional latent space: ; In the formula, The encoded modal feature vector has a dimension of 512. For random inactivation regularization, the probability is 0.3; For layer normalization operation, Let Gaussian error be the activation function of the linear unit. It is a linear transformation matrix. The original input data, where Including technical indicator eigenvectors h tech Fundamental eigenvectors h fund and text feature vectors h news .

[0010] Preferably, the feature dependencies within each modality feature vector are enhanced through a self-attention mechanism, including: For each modal eigenvector, i.e., the technical indicator eigenvector h tech Fundamental eigenvectors h fund and text feature vectors h news Self-attention calculations are performed separately. This self-attention mechanism is used to capture long-term dependencies between vectors and strengthen their internal temporal dependencies; specifically, it revolves around the feature vectors of technical indicators. h tech Perform self-attention calculation, fundamental feature vector h fund and text feature vectors h news Self-attention computation performs the same process; around the feature vector of technical indicators. h tech Performing self-attention calculations includes: ; in, MultiHeadAttn Multi-head attention mechanism; feature vectors of each modality , R For real numbers, T For time step, dhidden =12; Split the input modal feature vectors into multiple heads and calculate the attention weights for each head: ; in, AttentionFor attention function; Softmax For activation functions; Q For query vectors, K For key vectors, V The value vectors are all derived from technical indicator features. Generated through linear transformation; multi-head attention splits the dimension of the key vector. dk for: ; in, nhead The number of heads receiving multi-head attention, specifically: nhead =8; Then, the self-attention output will be... The features are added to the original features, and training is stabilized through layer normalization to ensure that the features retain the original information during updates, thus mitigating the gradient vanishing problem. Specifically: ; in, LayerNorm For layer normalization.

[0011] Preferably, in S2, intermodal feature interaction is achieved through a cross-modal attention mechanism in an alternating dominant modality manner, including: The cross-modal attention mechanism achieves multimodal interaction through the following formula: one of the three data classes is the primary modal feature, and the remaining two data classes are modal feature keys. The three data classes are alternately used as the primary modality, where the technical indicator feature vector... h tech Fundamental eigenvectors h fund and text feature vectors h news Perform the same process; this section focuses on the eigenvectors of technical indicators. h tech Main modality features, fundamental feature vectors h fund and text feature vectors h news For modal feature key values, the formula is: ; In the formula, The feature vector after cross-modal interaction. Main modal features These are modal key-value features.

[0012] Preferably, in S2, the main modality features and cross-modal interaction features are weighted and fused based on a dynamic gating mechanism, including: First, perform gating layer input splicing: ; in, gate input For feature vectors, concat This indicates that the three input vectors h tech , h cross , h tech - h cross Concatenate the components along their corresponding feature dimensions; residual difference term Used to capture feature offsets between modes; Then the gating is generated: ; In the formula, gate These are the learning weights, used for subsequent calculations of the weighted output. For the weights of the fully connected layer, For the Sigmoid function, To learn the temperature parameter, it is initialized to 1.0 and optimized through gradient descent. During model training, it is automatically optimized through backpropagation to dynamically adjust the gating strength. Final weighted output: ; Adaptive integration of cross-modal information was achieved through gated weighted fusion and residual learning; residual learning was used to preserve features processed by self-attention techniques. This mitigates the gradient vanishing problem through incremental updates of cross-modal features. LayerNorm Layer normalization standardizes the distribution of the fused features, stabilizing the training dynamics.

[0013] Preferably, in S3, the timing pooling includes: For each modality, the temporal dimension is averaged to generate a global aggregated feature, with a temporal step size of [missing value]. T =12, specifically: ; in, , , All are global aggregated features; Temporal pooling reduces model complexity, lowers the risk of overfitting, and improves computational performance. It also smooths time-series data, thereby enhancing the model's generalization ability.

[0014] Preferably, in S3, the fully connected fusion includes: The features obtained after temporal pooling are concatenated and then reduced in dimensionality using a two-layer fully connected network. ; in, hfused This represents the fused feature vector resulting from the fusion of multiple source features. W 1 and W Both 2 are basic vectors; , The final output is a fused feature vector. .

[0015] Preferably, the quantile regression layer outputs prediction results based on the quantile loss function, including quantile regression calculation and improved quantile loss, specifically: The quantile regression calculation addresses the uncertainty of quantified risk, and applies it to each quantile. Design includes GELU The two activated MLP layers: ; in, The target variable corresponding to each quantile, This is the first layer weight matrix. Here is the weight matrix for the second layer, where: ; ; The output dimension corresponds to the risk indicator of the prediction result; In the improved quantile loss, the loss function learns the variance parameter. Prediction error weighted by different quantiles: ; in, For the first The true value of each sample For the first The sample at the quantile q The predicted value below, For learning parameters; The total loss is the mean of the three quantile losses: ; in, L For the total loss, L q Quantile loss; pass Adaptively balancing the weights of different quantiles, the error of extreme quantiles is due to the exponential term. This is amplified, thereby increasing the model's sensitivity to tail risks.

[0016] The present invention also provides a prediction model constructed by the above-mentioned method of constructing a prediction model based on dynamic gating and cross-modal attention fusion, which is applied to the prediction of stock price crash risk.

[0017] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention provides a method for constructing a prediction model based on dynamic gating and cross-modal attention fusion, comprising: constructing a feature encoding layer to encode multi-source input data to obtain feature vectors for each modality, wherein the encoding process includes linear transformation, GELU activation, layer normalization, and Dropout regularization; constructing a multi-source data interaction layer to perform deep interaction on the feature vectors of each modality; wherein the deep interaction process includes strengthening the feature dependencies within each modality feature vector through a self-attention mechanism, then realizing inter-modal feature interaction through a cross-modal attention mechanism in an alternating master modality manner, and finally performing weighted fusion of master modality features and cross-modal interaction features based on a dynamic gating mechanism; constructing a feature fusion layer to perform temporal pooling and fully connected fusion on the multi-modal features after interaction to obtain a fused feature vector; and constructing a quantile regression layer to output prediction results based on the fused feature vector. This invention can deeply integrate multi-source heterogeneous data, accurately capture intra-modal and inter-modal feature dependencies, effectively quantify risk uncertainty, and adapt to the characteristics of financial data for stock price crash risk prediction. By integrating dynamic gating mechanism and cross-modal attention mechanism to construct prediction model, it solves the problems of existing prediction models in deep integration of multi-source heterogeneous data, cross-modal dynamic interaction and accurate quantification of tail risk, and thus achieves efficient prediction of stock price crash risk. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a method for constructing a prediction model based on dynamic gating and cross-modal attention fusion according to the present invention. Figure 2 A framework diagram of a prediction model based on dynamic gating and cross-modal attention fusion provided in an embodiment of the present invention; Figure 3 A feature coding layer framework diagram provided for embodiments of the present invention; Figure 4 This is a framework diagram of the multi-source data interaction layer provided in an embodiment of the present invention; Figure 5 This is a feature fusion layer framework diagram provided in an embodiment of the present invention; Figure 6 The quantile regression layer framework diagram provided for embodiments of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Example 1 like Figure 1 As shown, this embodiment provides a method for constructing a prediction model based on dynamic gating and cross-modal attention fusion, including the following steps: S1. Construct a feature encoding layer to encode the multi-source input data respectively to obtain the feature vectors of each modality. The encoding process includes linear transformation, GELU activation, layer normalization and Dropout regularization. S2. Construct a multi-source data interaction layer to perform deep interaction on the feature vectors of each modality. The deep interaction process includes strengthening the feature dependence within each modality feature vector through a self-attention mechanism, then realizing intermodal feature interaction through a cross-modal attention mechanism in an alternating master modality manner, and finally performing weighted fusion of master modality features and cross-modal interaction features based on a dynamic gating mechanism. S3. Construct a feature fusion layer to perform temporal pooling and fully connected fusion on the multimodal features after interaction to obtain a fused feature vector; S4. Construct a quantile regression layer and output the prediction result based on the fused feature vector.

[0023] like Figure 2 As shown, the prediction model provided in this embodiment adopts an end-to-end multimodal fusion architecture, which sequentially includes a feature encoding layer, a multi-source data interaction layer, a feature fusion layer, and a quantile regression layer, based on... Figure 2 As shown, multi-source input data (technical indicators, fundamentals, and text) are independently encoded and then deeply interact through self-attention and cross-modal attention. The feature dimensions are compressed through temporal pooling, and finally the risk prediction value is output by the quantile regression layer.

[0024] Specifically, refer to Figure 3 The original input data is first suppressed by a binary mask matrix, then mapped to a high-dimensional space through a linear transformation layer, nonlinearity is introduced by the GELU activation function, and finally a 512-dimensional feature vector is output through layer normalization and Dropout regularization.

[0025] In S1, before encoding the multi-source input data respectively, the following steps are included: The multi-source input data is suppressed for the influence of missing values ​​using a binary mask matrix: ; In the formula, A binary mask matrix is ​​used to mark data validity, i.e. Indicates non-missing, where, Indicates an index. t Indicates time; Indicates missing, dimension is N × T , N For the sample size, T For time step; This indicates broadcasting the mask matrix to the feature dimension; ⊙ indicates element-wise multiplication; It represents the outer product.

[0026] Furthermore, the multi-source input data is encoded separately, including: The feature encoding layer maps the processed multi-source input data to a high-dimensional latent space: ; In the formula, The encoded modal feature vector has a dimension of 512. For random inactivation regularization, the probability is 0.3; For layer normalization operation, Let Gaussian error be the activation function of the linear unit. It is a linear transformation matrix. The original input data, where Including technical indicator eigenvectors h tech Fundamental eigenvectors h fund and text feature vectors h news .

[0027] Furthermore, the feature dependencies within each modality's feature vector are enhanced through a self-attention mechanism, including: For each modal eigenvector, i.e., the technical indicator eigenvector h tech Fundamental eigenvectors h fund and text feature vectors h news Self-attention calculations are performed separately. This self-attention mechanism is used to capture long-term dependencies between vectors and strengthen their internal temporal dependencies; specifically, it revolves around the feature vectors of technical indicators.h tech Perform self-attention calculation, fundamental feature vector h fund and text feature vectors h news Self-attention computation, which performs the same process, specifically includes: ; ; ; in, MultiHeadAttn Multi-head attention mechanism; feature vectors of each modality , R For real numbers, T For timing step size, dhidden =12; Split the input modal feature vectors into multiple heads and calculate the attention weights for each head: ; in, Attention For attention function; Softmax For activation functions; Q For query vectors, K For key vectors, V The value vectors are all derived from technical indicator features. Generated through linear transformation; multi-head attention splits the dimension of the key vector. dk for: ; in, nhead The number of heads receiving multi-head attention, specifically: nhead =8; Then, the self-attention output will be... The features are added to the original features, and training is stabilized through layer normalization to ensure that the features retain the original information during updates, thus mitigating the gradient vanishing problem. Specifically: ; ; ; in, LayerNorm For layer normalization.

[0028] like Figure 4 As shown, a self-attention mechanism is used to strengthen the temporal dependencies within a single modality, a cross-modal attention mechanism is used to achieve feature interaction between modalities (with alternating dominant modalities as the query), and a dynamic gating mechanism (including temperature parameters) is used. τ Adaptive weighted fusion of main modality features and interaction features.

[0029] Specifically, using an alternating dominant modality approach, intermodal feature interaction is achieved through a cross-modal attention mechanism, including: The cross-modal attention mechanism achieves multimodal interaction through the following formula: one of the three data classes is the primary modal feature, and the remaining two data classes are modal feature keys. The three data classes are alternately used as the primary modality, where the technical indicator feature vector... h tech Fundamental eigenvectors h fund and text feature vectors h news Perform the same process, the specific formula is as follows: ; ; ; In the formula, The feature vector after cross-modal interaction. The two main modal features are used as the main modal features, and the other two types of data features besides the main modal features are concatenated to form the key-value features.

[0030] Furthermore, a weighted fusion of main modal features and cross-modal interaction features is performed based on a dynamic gating mechanism, including: First, perform gating layer input splicing: ; ; ; in, gate input For feature vectors, concat This means concatenating the three input vectors along their corresponding feature dimensions. The residual difference term is used to capture the feature offset between modes; Then the gating is generated: ; ; ; In the formula, gate These are the learning weights, used for subsequent calculations of the weighted output. For the weights of the fully connected layer, For the Sigmoid function, The temperature parameter is a learnable parameter, initialized to 1.0, and optimized through gradient descent. It is automatically optimized through backpropagation during model training to dynamically adjust the gating strength. Final weighted output: ; ; ; Adaptive integration of cross-modal information is achieved through gated weighted fusion and residual learning; residual learning is used to retain the three types of features after self-attention, and the gradient vanishing problem is alleviated through incremental updates of cross-modal features. LayerNorm Layer normalization standardizes the distribution of the fused features, stabilizing the training dynamics.

[0031] Reference Figure 5 The feature fusion layer first performs mean pooling on the interactive features along the temporal dimension. T =12), will T × d hidden Feature compression d hidden The multimodal pooling features are then concatenated, and the dimensionality is reduced by a two-layer fully connected network to output the final fused vector. h fused The diagram highlights the balance between feature compression and information preservation in the pooling layer symbols and fully connected layer arrows. Specifically, in S3, the temporal pooling includes: For each modality, the temporal dimension is averaged to generate a global aggregated feature, with a temporal step size of [missing value]. T =12, specifically: ; in, , , All are global aggregated features; Temporal pooling reduces model complexity, lowers the risk of overfitting, and improves computational performance. It also smooths time-series data, thereby enhancing the model's generalization ability.

[0032] Furthermore, the fully connected fusion includes: The features obtained after temporal pooling are concatenated and then reduced in dimensionality using a two-layer fully connected network: ; in, h fused This represents the fused feature vector resulting from the fusion of multiple source features. W 1 and W Both 2 are basic vectors; , The final output is a fused feature vector. .

[0033] Furthermore, such as Figure 6 As shown, fusion features h fusedInput three independent MLPs, each with a two-layer fully connected quantile predictor. , With GELU activation; the loss function is passed through learnable parameters log. σ q Adaptive weighted extreme quantile error. Specifically, the quantile regression layer outputs prediction results based on the quantile loss function, including quantile regression calculation and improved quantile loss, specifically: The quantile regression calculation addresses the uncertainty of quantified risk, and applies it to each quantile. Design includes GELU The two activated MLP layers: ; in, The target variable corresponding to each quantile, This is the first layer weight matrix. Here is the weight matrix for the second layer, where: ; ; The output dimension corresponds to the risk indicator of the prediction result; In the improved quantile loss, the loss function learns the variance parameter. Prediction error weighted by different quantiles: ; in, For the first The true value of each sample For the first The sample at the quantile q The predicted value below, These are learnable parameters; The total loss is the mean of the three quantile losses: ; in, L For the total loss, L q Quantile loss; pass Adaptively balancing the weights of different quantiles, the error of extreme quantiles is due to the exponential term. This is amplified, thereby increasing the model's sensitivity to tail risks.

[0034] Example 2 This embodiment, based on the prediction model constructed in Embodiment 1 using dynamic gating and cross-modal attention fusion, uses A-share listed companies in my country from 2010 to 2022 as research samples to verify the actual effect of the model in predicting stock price crash risk. Data comes from the Guotai An Financial Database, including fundamental data such as financial news, financial indicators (e.g., debt-to-equity ratio, ROA), and governance indicators (e.g., equity checks and balances), as well as market trading technical indicators and trading activity data such as Average True Range (ATR) and Momentum Index (MOM). The sample is divided into a training set (January 2010 – December 2018, 9 years, 117,740 company-monthly observations), a validation set (January 2019 – December 2020, 2 years, 26,387 observations), and a test set (January 2021 – December 2022, 2 years, 24,156 observations).

[0035] The forecasting objective uses the Negative Return Skewness Coefficient (NCSKEW) and the Return Up-Down Volatility Ratio (DUVOL) as proxy indicators for stock market crash risk. NCSKEW measures the negative skewness of the return distribution, while DUVOL characterizes the difference in volatility between price increases and decreases. An out-of-sample R-squared is used. 2 The value is used to evaluate the predictive performance, and the formula is: ; in, For the true value, For predicted values, R is the mean of the true values. 2 A larger value indicates better prediction performance. Linear regression, Lasso regression, Random Forest (RF), Gradient Boosting Tree (LGBM), LSTM, and the basic Transformer (without dynamic gating) were selected as comparison models. Model training used the Adam optimizer with an initial learning rate of 1e-4, a batch size of 128, and a maximum training epoch of 100. Overfitting was avoided using the EarlyStopping mechanism. Input features were standardized, and time-series inputs used a sliding window method with a time step of T=12.

[0036] Experimental results on the test set show that the prediction model provided in Example 1 significantly outperforms other models in predicting both types of metrics, particularly in predicting the R-value of NCSKEW. 2 Reaching 23.35%, for DUVOL's R 2 The R² rate reached 17.95%, representing improvements of 32.8% and 38.1% respectively compared to the basic Transformer, indicating that dynamic gating and cross-modal attention mechanisms effectively captured the nonlinear interactions of multi-source data. In contrast to models using only a single data source, the prediction module provided in Example 1, after fusing three types of data, achieved a higher R² rate. 2An average improvement of 8.5% was achieved, validating the beneficial effect of multi-source information synergy on risk prediction. For the high-risk sample in the NCSKE Wtop 5%, the prediction model provided in Example 1 showed an R-value... 2 The quantile regression layer achieved a 31.2% accuracy rate, a 66.8% improvement over the LGBM model, indicating that it has a stronger ability to characterize extreme crash risks. SHAP value analysis revealed that technical indicators (such as ATR and Beta) and news sentiment characteristics (such as the proportion of negative sentiment) contributed the most to the prediction, accounting for 35% and 28% respectively. After introducing the interaction layer, the importance of company fundamental indicators (such as the debt-to-equity ratio) increased by 12%, indicating that the prediction model provided in Example 1 captured the risk amplification effect of highly leveraged companies plus negative news.

[0037] In summary, this embodiment verifies the effectiveness of the prediction model based on dynamic gating and cross-modal attention fusion constructed in Embodiment 1 in predicting stock price crash risk. By integrating dynamic gating and cross-modal attention mechanisms, the model achieves deep interaction of multi-source heterogeneous data, improves sensitivity to extreme risks, and provides reliable technical support for financial market risk early warning.

[0038] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0039] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for constructing a prediction model based on dynamic gating and cross-modal attention fusion, characterized in that, Includes the following steps: S1. Construct a feature encoding layer to encode the multi-source input data respectively to obtain the feature vectors of each modality. The encoding process includes linear transformation, GELU activation, layer normalization and Dropout regularization. S2. Construct a multi-source data interaction layer to perform deep interaction on the feature vectors of each modality. The deep interaction process includes strengthening the feature dependence within each modality feature vector through a self-attention mechanism, then realizing intermodal feature interaction through a cross-modal attention mechanism in an alternating master modality manner, and finally performing weighted fusion of master modality features and cross-modal interaction features based on a dynamic gating mechanism. S3. Construct a feature fusion layer to perform temporal pooling and fully connected fusion on the multimodal features after interaction to obtain a fused feature vector; S4. Construct a quantile regression layer and output the prediction result based on the fused feature vector.

2. The method for constructing a prediction model based on dynamic gating and cross-modal attention fusion according to claim 1, characterized in that, In S1, before encoding the multi-source input data respectively, the following steps are included: The multi-source input data is suppressed for the influence of missing values ​​using a binary mask matrix: ； In the formula, A binary mask matrix is ​​used to mark data validity, i.e. Indicates non-missing, where, Indicates an index. t Indicates time; Indicates missing, dimension is N × T , N For the sample size, T For time step; This indicates broadcasting the mask matrix to the feature dimension; ⊙ indicates element-wise multiplication; It represents the outer product.

3. The method for constructing a prediction model based on dynamic gating and cross-modal attention fusion according to claim 2, characterized in that, In S1, the multi-source input data is encoded respectively, including: The feature encoding layer maps the processed multi-source input data to a high-dimensional latent space: ； In the formula, The encoded modal feature vector has a dimension of 512. For random inactivation regularization, the probability is 0.3; For layer normalization operation, Let Gaussian error be the activation function of the linear unit. It is a linear transformation matrix. The original input data, where Including technical indicator eigenvectors h tech Fundamental eigenvectors h fund and text feature vectors h news .

4. The method for constructing a prediction model based on dynamic gating and cross-modal attention fusion according to claim 3, characterized in that, Strengthening feature dependencies within feature vectors of each modality through self-attention mechanisms, including: For each modal eigenvector, i.e., the technical indicator eigenvector h tech Fundamental eigenvectors h fund and text feature vectors h news Self-attention calculations are performed separately. This self-attention mechanism is used to capture long-term dependencies between vectors and strengthen their internal temporal dependencies; specifically, it revolves around the feature vectors of technical indicators. h tech Perform self-attention calculation, fundamental feature vector h fund and text feature vectors h news Self-attention computation performs the same process; around the feature vector of technical indicators. h tech Performing self-attention calculations includes: ； in, MultiHeadAttn Multi-head attention mechanism; feature vectors of each modality , R For real numbers, T For time step, dhidden =12; Split the input modal feature vectors into multiple heads and calculate the attention weights for each head: ； in, Attention For attention function; Softmax For activation functions; Q For query vectors, K For key vectors, V The value vectors are all derived from technical indicator features. Generated through linear transformation; multi-head attention splits the dimension of the key vector. dk for: ； in, nhead The number of heads receiving multi-head attention, specifically: nhead =8; Then, the self-attention output will be... The features are added to the original features, and training is stabilized through layer normalization to ensure that the features retain the original information during updates, thus mitigating the gradient vanishing problem. Specifically: ； in, LayerNorm For layer normalization.

5. The method for constructing a prediction model based on dynamic gating and cross-modal attention fusion according to claim 4, characterized in that, In S2, intermodal feature interaction is achieved through a cross-modal attention mechanism using an alternating dominant modality approach, including: The cross-modal attention mechanism achieves multimodal interaction through the following formula: one of the three data classes is the primary modal feature, and the remaining two data classes are modal feature keys. The three data classes are alternately used as the primary modality, where the technical indicator feature vector... h tech Fundamental eigenvectors h fund and text feature vectors h news Perform the same process; this section focuses on the eigenvectors of technical indicators. h tech Main modality features, fundamental feature vectors h fund and text feature vectors h news For modal feature key values, the formula is: ； In the formula, The feature vector after cross-modal interaction. Main modal features These are modal key-value features.

6. The method for constructing a prediction model based on dynamic gating and cross-modal attention fusion according to claim 5, characterized in that, In S2, a weighted fusion of main modal features and cross-modal interaction features is performed based on a dynamic gating mechanism, including: First, perform gating layer input splicing: ； in, gate input For feature vectors, concat This indicates that the three input vectors h tech , h cross , h tech - h cross Concatenate the components along their corresponding feature dimensions; residual difference term Used to capture feature offsets between modes; Then the gating is generated: ； In the formula, gate These are the learning weights, used for subsequent calculations of the weighted output. For the weights of the fully connected layer, For the Sigmoid function, To learn the temperature parameter, it is initialized to 1.0 and optimized through gradient descent. During model training, it is automatically optimized through backpropagation to dynamically adjust the gating strength. Final weighted output: ； Adaptive integration of cross-modal information was achieved through gated weighted fusion and residual learning; residual learning was used to preserve features processed by self-attention techniques. This mitigates the gradient vanishing problem through incremental updates of cross-modal features. LayerNorm Layer normalization standardizes the distribution of the fused features, stabilizing the training dynamics.

7. The method for constructing a prediction model based on dynamic gating and cross-modal attention fusion according to claim 6, characterized in that, In S3, the time-series pooling includes: For each modality, the temporal dimension is averaged to generate a global aggregated feature, with a temporal step size of [missing value]. T =12, specifically: ； in, , , All are global aggregated features; Temporal pooling reduces model complexity, lowers the risk of overfitting, and improves computational performance. It also smooths time-series data, thereby enhancing the model's generalization ability.

8. The method for constructing a prediction model based on dynamic gating and cross-modal attention fusion according to claim 7, characterized in that, In S3, the fully connected fusion includes: The features obtained after temporal pooling are concatenated and then reduced in dimensionality using a two-layer fully connected network. ； in, h fused This represents the fused feature vector resulting from the fusion of multiple source features. W 1 and W Both 2 are basic vectors; , The final output is a fused feature vector. .

9. The method for constructing a prediction model based on dynamic gating and cross-modal attention fusion according to claim 8, characterized in that, The quantile regression layer outputs prediction results based on the quantile loss function, including quantile regression calculation and improved quantile loss, specifically: The quantile regression calculation addresses the uncertainty of quantified risk, and applies it to each quantile. Design includes GELU The two activated MLP layers: ； in, The target variable corresponding to each quantile, This is the first layer weight matrix. Here is the weight matrix for the second layer, where: ； ； The output dimension corresponds to the risk indicator of the prediction result; In the improved quantile loss, the loss function learns the variance parameter. Prediction error weighted by different quantiles: ； in, For the first The true value of each sample For the first The sample at the quantile q The predicted value below, For learning parameters; The total loss is the mean of the three quantile losses: ； in, L For the total loss, L q Quantile loss; pass Adaptively balancing the weights of different quantiles, the error of extreme quantiles is due to the exponential term. This is amplified, thereby increasing the model's sensitivity to tail risks.

10. A prediction model constructed using the method for constructing a prediction model based on dynamic gating and cross-modal attention fusion according to any one of claims 1 to 9, characterized in that, It is used to predict the risk of stock price crashes.

Images