A financial risk prediction method based on multi-modal data fusion

Abstract

The application provides a financial risk prediction method based on multi-modal data fusion, and belongs to the technical field of artificial intelligence risk prediction, and comprises the following steps: S1, audio feature vector extraction is performed on a financial report telephone conference of a target company; S2, text summary feature vector extraction is performed on the financial report telephone conference of the target company; S3, a large language model is used to summarize the text summary of the financial report telephone conference of the target company, and an embedding vector corresponding to the summary of the text summary is obtained; S4, news text features of the target company are extracted, and a news text feature vector is obtained; S5, feature extraction is performed on time series transaction data of the target company in a period before a target date, and a time series data feature vector is obtained; S6, a joint representation vector is obtained; and S7, the joint representation vector is input into a multi-task learning framework to perform risk index prediction. The application has the advantages of significantly improved prediction accuracy, comprehensive utilization of implicit information, and good multi-task learning prediction capability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence risk prediction, and in particular to a financial risk prediction method based on multi-modal data fusion. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, the intelligent level of the financial industry has been significantly improved. In the field of financial risk prediction, artificial intelligence technology not only improves the efficiency of risk identification, but also provides important support for enterprises to optimize investment decisions and improve risk management. However, the current research and practice still have some deficiencies in how to efficiently integrate multi-modal data to achieve more accurate financial risk prediction. Existing financial risk prediction technologies mainly focus on the use of single data sources, such as historical stock price analysis based on time series or news text sentiment analysis based on natural language processing. Although this method is effective in specific scenarios, it is difficult to fully reflect the multi-dimensional driving factors of market fluctuations. Especially in the face of complex market environment and growing data diversity, the limitations of single data source prediction methods become increasingly apparent. In addition, a large amount of key information in the financial market exists in the form of unstructured data, such as text summaries, voice recordings, and media news reports of conference calls. These data contain implicit features such as the speaker's tone, speed, and emotional changes, which are important clues for assessing market risk. However, traditional models have technical bottlenecks when dealing with unstructured data, and lack effective fusion capabilities for multi-modal data, making it difficult for prediction results to fully and accurately reflect market dynamics. The existing technology has the following significant defects in the field of financial risk prediction: the limitation of prediction results due to single data source, and the poor adaptability and stability of the prediction model, which greatly affects the accuracy and reliability of the prediction results.

[0003] In recent years, large language models (LLM) have gradually been introduced into the financial field due to their advantages in cross-domain text processing, sentiment analysis, and multi-task learning. Studies have shown that large language models can efficiently process long texts and generate high-quality summaries, and have the ability to analyze financial data in depth. However, relying solely on a single model is difficult to fully capture the multi-dimensional dynamics of the financial market. How to integrate multi-modal data into a unified prediction framework and combine the advantages of large language models for in-depth analysis has become a core problem that needs to be solved in the current field of financial risk prediction. In summary, a financial risk prediction method based on multi-modal data fusion is provided, which comprehensively utilizes multi-modal data and performs deep feature extraction and fusion analysis through a large language model. Through the integration of multi-source data and the design of a multi-task prediction framework, the present application significantly improves the accuracy and stability of financial risk prediction, providing strong technical support for financial risk management, and is therefore necessary. SUMMARY

[0004] In view of this, the present invention proposes a financial risk prediction method based on multimodal data fusion that integrates multiple data sources, comprehensively captures the dynamics and market information of target companies, predicts multiple risk indicators, and has good predictive comprehensiveness and accuracy.

[0005] The technical solution of this invention is implemented as follows: This invention provides a financial risk prediction method based on multimodal data fusion, comprising the following steps:

[0006] S1: Extract audio feature vectors from the target company's earnings call;

[0007] S2: Extract feature vectors from the text transcripts of the target company's earnings call;

[0008] S3: Use a large language model to summarize the text transcripts of the target company's earnings call and obtain the embedding vector corresponding to the summary of the text transcripts.

[0009] S4: Extract news text features from the target company to obtain news text feature vectors;

[0010] S5: Extract features from the time-series transaction data of the target company during a period before the target date to obtain the feature vector of the time-series data;

[0011] S6: Fuse the obtained audio feature vector, text summary feature vector, embedding vector corresponding to the text summary, news text feature vector, and time-series data feature vector to obtain a joint representation vector; and

[0012] S7: Input the joint representation vector into the multi-task learning framework to predict risk indicators.

[0013] Based on the above technical solutions, the preferred embodiment of step S1 is as follows:

[0014] S11: Use the WenetSpeech pre-trained language model to extract the embedding vectors from the earnings call audio. For a pre-processed audio segment... in Let i represent the i-th frame of the audio, and n represent the number of frames in the audio. Convert each frame of audio into a vector representation. Thus, the entire audio segment A is obtained. c audio embedding vector

[0015] S12: Embed the audio vector E ac The audio feature vector T is then fed into a multi-head self-attention module (MHSA) to further extract the audio feature vector T. ac =MHSA(E ac );

[0016] S13: Transfer the feature vector T ac The compressed audio feature vector T is obtained by feeding it into an average pooling layer. a T a =AveragePooling(T ac ).

[0017] Preferably, the vector representation of each frame of audio All dimensions are 512, and the compressed audio feature vector T a The dimension of the text summary feature vector is 512; the dimension of the text summary feature vector is 768; the dimension of the embedding vector corresponding to the summary of the text summary is 768; and the dimension of the joint representation vector is 512.

[0018] Preferably, the Multi-Head Self-Attention (MHSA) module uses multiple attention heads to process the input audio embedding vector in parallel, and calculates the attention weights of the audio embedding vector using the following formula: Where the query vector Q = E ac W Q The key vector K = E ac W K Value vector V = E ac W V W Q W K and W V Let d be a linear projection matrix. k The dimension of the attention heads is defined, and the softmax function is used for normalization. The outputs of each attention head are concatenated, and a linear transformation is applied to obtain the audio feature vector T. ac T ac =Concat(head1,head2,head) h W o Where subscripts 1, 2, ..., h are distinguishing markers for different attention heads, W o Let be the linear transformation matrix after merging, and Concat be the merging function.

[0019] Further preferred, the specific content of step S2 is as follows:

[0020] S21: Preprocess the transcript of the target company's earnings call to obtain a set of sentences. in Let L represent the L-th sentence in the text summary, where L = 1, 2, ..., l, and l represents the number of sentences in the text summary; using the Sentence-BERT pre-trained language model, each sentence... Mapped to the embedding vector of the earnings call transcript Obtain the vector representation of the entire earnings call transcript.

[0021] S22: The vector representation of the entire earnings call transcript is fed into the multi-head self-attention module MHSA to further extract the feature vector T of the transcript. tc T tc =MHSA(E tc );

[0022] S23: Transfer the feature vector T of the text summary tc The data is fed into an Average Pooling Layer to obtain the compressed text summary feature vector T. t =AveragePooling(T tc ).

[0023] Furthermore, step S3 specifically includes:

[0024] S31: Paragraph segmentation. The transcript of the target company's earnings call is segmented into logical paragraphs to obtain paragraph p. M =p1,p2,,p m M = 1, 2, ..., m, where the subscript m represents the number of paragraphs; for each paragraph p... M Input a large language model (LLM) to extract paragraph-level summaries. M =LLM(p M ), s M It is paragraph p M Abstract;

[0025] S32: Summarize all paragraphs {s1,s2,s...} M The data is merged into a single text and input into a large language model to generate a comprehensive summary S, where S = LLM({s1,s2,s...). M});

[0026] S33: Use the Sentence-BERT pre-trained language model to vectorize the comprehensive summary S, generating the corresponding embedding vector T for the summary of the text minutes. l T l =SBERT(S).

[0027] In a further preferred embodiment, step S4 specifically includes:

[0028] S41: Collect news text data of the target company several days before the target transaction date, represented as N = {n1, n2, n...} k}, where n K0 This represents the K0th news article, where k is the total number of news articles.

[0029] S42: Use a Large Language Model (LLM) for each news item n K0 Perform parsing and extract metadata m K0 m K0 =LLM(n K0 ); Integrate the metadata of all news articles into a single metadata set M. N ={m1,m2,,m k};

[0030] S43: Find the k historical news groups N in the historical news dataset that are most similar to the news text data N from several days prior to the target transaction date in terms of metadata. K0 =N1,N2,,N k Calculate the relationship between N and any set of historical news groups N K0 semantic relevance, Where f is a function that converts news text into an embedded vector, and selects a set of historical news H that has the highest semantic similarity to news text data N from several days before the target trading day;

[0031] S44: Concatenate the text related to N, H, and subsequent market trends after H, and then use the SBERT model to convert the concatenated text into an embedding vector T. n Embedded vector T n This serves as a feature vector of news texts from the target company several days prior to the target transaction date.

[0032] Furthermore, step S5 specifically includes the following:

[0033] S51: Collect the target company's time-series transaction data D for the 30 days prior to the target date, including daily closing prices and trading volumes, represented as D = {(p1, v1), (p2, v2), ..., (p...}. d ,v d )}, where p F =p1,p2,,p d v represents the closing price on day F. F =v1,v2,,v d This represents the trading volume on day F.

[0034] S52: Input the time-series transaction data D into the Bi-Short Memory (Bi-LSTM) network. The Bi-LSTM network captures the temporal dynamics of the transaction data and outputs a feature vector T containing the time-series data. v T v =BiLSTM(D);

[0035] S53: Capturing the dynamic relationships between different time-series features using a vector autoregressive (VAR) model:

[0036] log(σ 3,t )=α3+β 1,1 log(σ -3,t )+β 1,2 log(σ -7,t )+β 1.3 log(σ -15,t )+β 1.4. log(σ -30,t )+u 3,t ;

[0037] log(σ 7,t )=α7+β 2,1 log(σ -3,t )+β 2,2 log(σ -7,t )+β 2.3 log(σ -15,t )+β 2.4. log(σ -30,t )+u 7,t ;

[0038] log(σ 15,t )=α 15 +β 3,1 log(σ -3,t )+β 3,2 log(σ -7,t )+β 3.3 log(σ -15,t )+β 3.4. log(σ -30,t )+u 15,t ;

[0039] log(σ 30,t )=α 30 +β 4,1 log(σ -3,t )+β 4,2 log(σ -7,t )+β 4.3 log(σ -15,t )+β 4.4. log(σ -30,t )+u 30,t ;

[0040] σ z,t This represents the volatility of the target company's stock price over the days z = 3, 7, 15, and 30, where z ≤ F and z ∈ t; u z,t It is a white noise term; β a,b It is a coefficient matrix of dynamic relationships, a,b=1,2,3,4; α z This is the intercept term; stock price volatility is defined as the standard deviation of the target company's returns over z days:

[0041] In a further preferred embodiment, the joint representation vector E described in step S6 is fused using the following formula: E = w0 + w1T a +w2T t +w3T l +w4T n +w5T v +ε, where w0 is the bias term, w1, w2, w3, w4, w are the 5-fold fusion weights, and ε is the error term, representing random noise.

[0042] More preferably, step S7 involves using the joint representation vector E generated in step S6 as input to a multi-task learning framework, which simultaneously predicts the following risk indicator: the volatility σ of the target company's stock price over time spans of 3 days, 7 days, 15 days, and 30 days. 3,t σ 7,t σ 15,t and σ 30,t And the daily risk value VAR;

[0043] For each time span, an independent first prediction subnetwork (MLP) is constructed to predict stock price volatility. The prediction results for stock price volatility are as follows: in f represents the predicted volatility. MLP,z (·) represents the first prediction subnetwork MLP with a time span of z;

[0044] For the daily risk value (VAR), an independent second prediction subnetwork (MLP) is constructed, with the joint representation vector E as input and the predicted VAR value as output. in f is the predicted value of VAR; MLP,VAR (·) represents the second prediction subnetwork MLP;

[0045] The joint loss function simultaneously optimizes both stock price volatility and VAR prediction tasks. The formula for the joint loss function is: μ is a weighted hyperparameter that balances the prediction error of stock price volatility and the prediction error of VAR; y j and It represents the true and predicted values ​​of the stock price volatility indicator; The mean squared error of stock price volatility forecast; q represents the quantile threshold of daily risk value; V and These represent the actual daily risk value and the predicted daily risk value, respectively. This represents the quantile regression loss function for the daily risk value prediction task.

[0046] The financial risk prediction method based on multimodal data fusion provided by this invention has the following advantages compared with the prior art:

[0047] (1) This invention constructs a new financial risk prediction framework by integrating multimodal data, including audio and text of financial statement conference calls, news text and time series data. This multimodal feature fusion method significantly improves the ability to perceive financial market risks, can capture multi-level information on company operations and market environment, and solves the problem of insufficient prediction accuracy caused by a single data source in existing technologies.

[0048] (2) Existing technologies often overlook the non-explicit features contained in the audio of earnings call conference calls, such as the speaker’s tone, speaking speed and emotions. This invention introduces the multi-head self-attention mechanism MHSA and the large language model LLM to jointly extract and deeply analyze the features of audio and text data, which can reveal the potential information correlations that are difficult to capture by traditional text analysis methods, thereby improving the accuracy and insight of prediction.

[0049] (3) This invention employs a multi-task learning framework, which can simultaneously predict multiple risk indicators, including stock price volatility over different time spans and daily risk value (VAR). This technical solution not only improves the model's prediction efficiency but also effectively alleviates the dependence of traditional methods on a single task, enhancing the model's adaptability to complex task scenarios. Furthermore, by introducing a joint loss function, the multi-task learning process is optimized, further improving prediction performance. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating a financial risk prediction method based on multimodal data fusion according to the present invention. Detailed Implementation

[0052] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0053] Existing financial risk forecasting methods mostly rely on a single data source, failing to capture the complex relationship between internal corporate operations and the external market environment. This results in incomplete and inaccurate forecasts, and the inability to simultaneously predict multiple risk indicators. Therefore, if... Figure 1 As shown, this invention provides a financial risk prediction method based on multimodal data fusion, comprising the following steps:

[0054] S1: Extract audio feature vectors from the target company's earnings call.

[0055] The specific content of step S1 is as follows:

[0056] S10: Audio preprocessing. The earnings call audio is segmented into frames, with each frame having a fixed length, such as 25 milliseconds, and a 10-millisecond overlap between adjacent frames to ensure the capture of the temporal features of continuous speech. Then, the earnings call audio is normalized to standardize the waveform amplitude and eliminate the influence of recording equipment or volume differences. The earnings call audio format is converted, such as WAV or FLAC, for subsequent feature extraction.

[0057] S11: Use the WenetSpeech pre-trained language model to extract the embedding vectors from the earnings call audio. For a pre-processed audio segment... in Let i represent the i-th frame of the audio, and n represent the number of frames in the audio. Convert each frame of audio into a vector representation. Thus, the entire audio segment A is obtained. c audio embedding vector The pre-trained language model WenetSpeech uses a sampled Transformer architecture. This model has been trained on large-scale speech datasets and can effectively extract hidden features of speech, making it suitable for the field of natural language processing.

[0058] S12: Embed the audio vector E ac The audio feature vector T is then fed into a multi-head self-attention module (MHSA) to further extract the audio feature vector T. ac =MHSA(E ac Multi-head self-attention is a common technique in this field. It typically uses 8 or 16 attention heads to process the input audio embedding vector in parallel, with each attention head focusing on the global dependencies between audio frames in a different way.

[0059] The Multi-Head Self-Attention (MHSA) module uses multiple attention heads to process the input audio embedding vector in parallel, and calculates the attention weights of the audio embedding vector using the following formula: Where the query vector Q = E ac W Q The key vector K = E ac W K Value vector V = E ac W V W Q W K and W V Let d be a linear projection matrix. k The dimension of the attention heads is defined, and the softmax function is used for normalization. The outputs of each attention head are concatenated, and a linear transformation is applied to obtain the audio feature vector T. ac T ac =Concat(head1,head2,head) h W o Where subscripts 1, 2, ..., h are distinguishing markers for different attention heads, W o Let be the linear transformation matrix after merging, and Concat be the merging function.

[0060] S13: Transfer the feature vector T ac The compressed audio feature vector T is obtained by feeding it into an average pooling layer. a T a =AveragePooling(T ac ).

[0061] To retain a sufficient amount of feature information, as a preferred implementation, in this invention, the vector representation of each frame of audio... All dimensions are 512, and the compressed audio feature vector T a The dimension is also 512; for the feature vector T of the audio... ac The average value of each feature dimension is taken and compressed into a single 512-dimensional vector, which is the result of the average pooling layer operation. This step effectively reduces the feature dimension while preserving global speech information, resulting in a compressed audio feature vector T. a It can represent the implicit information such as tone, emotion, and speech rate contained in audio data, providing high-quality input for subsequent multimodal feature fusion.

[0062] In addition, the dimension of the text summary feature vector is 768; the dimension of the embedding vector corresponding to the summary of the text summary is 768; and the dimension of the joint representation vector is 512.

[0063] Step S1 of this invention fully utilizes the unstructured data characteristics of earnings call audio to extract high-dimensional, information-dense audio features, thereby providing important support for financial risk prediction models.

[0064] S2: Extract the feature vector of the text transcript of the target company's earnings call.

[0065] The specific content of step S2 is as follows:

[0066] S21: Preprocess the transcript of the target company's earnings call to obtain a set of sentences. Where t c L This represents the L-th sentence in the text summary, where L = 1, 2, ..., l, and l represents the number of sentences in the text summary; the Sentence-BERT pre-trained language model, abbreviated as SBERT, is used to process each sentence... Mapped to a 768-dimensional embedding vector of the earnings call transcript Obtain the vector representation of the entire earnings call transcript. SBERT is a sentence-level semantic representation model based on the BERT architecture. It optimizes the ability to capture semantic similarity between sentences by comparing and learning sentence pairs through the Siamese network.

[0067] The text preprocessing involves standardizing the transcripts of the target company's earnings call, removing redundant characters, unifying the format, and eliminating meaningless stop words such as "um" and "uh." Then, sentences are divided according to punctuation marks such as periods and semicolons in the transcripts to obtain a sentence set.

[0068] S22: The vector representation of the entire earnings call transcript is fed into the multi-head self-attention module MHSA to further extract the feature vector T of the transcript. tc T tc =MHSA(E tc The Multi-Head Self-Attention Module (MHSA) mentioned here is similar to the module in step S12, but its function is to capture global dependencies between sentences. It uses multiple attention heads, each focusing on different semantic relationships. After concatenating the outputs of each attention head, a linear transformation is performed to generate the final context-enhanced text summary feature vector T. tc .

[0069] S23: Transfer the feature vector T of the text summary tc The data is fed into an Average Pooling Layer to obtain the compressed text summary feature vector T. t =AveragePooling(T tc ).

[0070] The feature vector T of the text summary tcThe average value of each feature dimension is compressed into a single 768-dimensional text summary feature vector, representing the comprehensive semantic information of the entire earnings call transcript. This fully preserves the semantics and contextual relationships in the text, including key financial information, semantic sentiment, and logical connections between sentences. This provides high-quality text input for subsequent multimodal feature fusion. This invention has significant advantages in capturing implicit textual information from earnings call transcripts by combining contextual features, providing an important data foundation for financial risk prediction.

[0071] S3: Use a large language model to summarize the text transcript of the target company's earnings call and obtain the embedding vector corresponding to the summary of the text transcript. This step extracts the core content by segmenting and summarizing the text transcript of the earnings call and generates the embedding vector corresponding to the summary of the text transcript for subsequent use.

[0072] The specific content of step S3 is as follows:

[0073] S31: Paragraph segmentation. The transcript of the target company's earnings call is segmented into logical paragraphs to obtain paragraph p. M =p1,p2,,p m M = 1, 2, ..., m, where the subscript m represents the number of paragraphs; for each paragraph p... M Input a large language model (LLM) to extract paragraph-level summaries. M =LLM(p M ), s M It is paragraph p M Abstract.

[0074] S32: Summarize all paragraphs {s1,s2,s...} M The data is merged into a single text and input into a large language model to generate a comprehensive summary S, where S = LLM({s1,s2,s...). M After the summary is generated, the granularity of the summary can be further adjusted according to needs, such as controlling the length of longer text summaries or removing redundant information.

[0075] S33: Use the Sentence-BERT pre-trained language model to vectorize the comprehensive summary S, generating the corresponding embedding vector T for the summary of the text minutes. l T l =SBERT(S). The summary of the text minutes corresponds to the embedding vector T. l This is a 768-dimensional feature vector representing the semantic information summarized in the text minutes. This vector integrates the core content and global information of the financial meeting text, providing important support for multimodal feature fusion.

[0076] Through the above steps, the earnings call transcript, after being processed through paragraph segmentation, paragraph-level summarization, and comprehensive summary, generates an embedding vector T. l It effectively summarizes the key content of the text, which can significantly improve the efficiency of financial risk prediction models in utilizing text data and the accuracy of prediction.

[0077] S4: Extract features from the news text of the target company to obtain a news text feature vector. This step generates a news text feature vector for the target company's relevant news text by performing semantic analysis and similarity calculation.

[0078] The specific content of step S4 is as follows:

[0079] S41: Collect news text data of the target company several days before the target transaction date, represented as N = {n1, n2, n...} k}, where n K0 This represents the K0th news article, where k is the total number of news articles.

[0080] S42: Use a Large Language Model (LLM) for each news item n K0 Perform parsing and extract metadata m K0 m K0 =LLM(n K0 The extracted metadata includes sentiment (positive, negative, or neutral), financial metrics mentioned in the news (such as revenue and profit), and other key information relevant to the target company. All news metadata is integrated into a single metadata set M. N ={m1,m2,,m k}

[0081] S43: Find the k historical news groups N in the historical news dataset that are most similar to the news text data N from several days prior to the target transaction date in terms of metadata. K0 =N1,N2,,N k Calculate the relationship between N and any set of historical news groups N K0 semantic relevance, Where f is a function that converts news text into an embedded vector, and selects a set of historical news H that has the highest semantic similarity to news text data N from several days before the target trading day.

[0082] S44: Concatenate the text related to N, H, and subsequent market trends after H, and then use the SBERT model to convert the concatenated text into an embedding vector T. n Embedded vector T n This serves as a feature vector of news texts from the target company several days prior to the target transaction date.

[0083] This corresponds to text from three sources: 1. News text data N from several days prior to the target trading day; 2. Historical news H; 3. Market trend-related text after historical news H occurred, i.e., news after historical news H occurred. For example, historical news on January 1 mentioned that Company A had a debt crisis, and a news article on January 2 mentioned that Company A's stock price plummeted. The news article on January 2 is the market trend-related text after historical news H occurred.

[0084] Through step S4 above, the generated news text feature vector T of the target company several days prior to the target transaction date is obtained. n It effectively integrates the semantic information of the target news, sentiment analysis results, and similarity with historical news, providing high-quality input features for subsequent multimodal feature fusion.

[0085] S5: Extract features from the target company's time-series transaction data for a period preceding the target date to obtain the feature vector of the time-series data. As a preferred implementation, this step processes the target company's time-series transaction data for the 30 days preceding the target date to extract key features and capture the dynamic relationships between different time-series features.

[0086] The specific content of step S5 is as follows:

[0087] S51: Collect the target company's time-series transaction data D for the 30 days prior to the target date, including daily closing prices and trading volumes, represented as D = {(p1, v1), (p2, v2), ..., (p...}. d ,v d )}, where p F =p1,p2,,p d v represents the closing price on day F. F =v1,v2,,v d This represents the trading volume on day F.

[0088] S52: Input the time-series transaction data D into the Bi-Short Memory (Bi-LSTM) network. The Bi-LSTM network captures the temporal dynamics of the transaction data and outputs a 128-dimensional feature vector T containing the time-series data. v T v =BiLSTM(D); Bi-LSTM, a bidirectional long short-term memory network, models time-series data and captures the time-series dynamic characteristics of transaction data.

[0089] S53: Capturing the dynamic relationships between different time-series features using a vector autoregressive (VAR) model:

[0090] log(σ 3,t )=α3+β 1,1 log(σ -3,t )+β1,2 log(σ -7,t )+β 1.3 log(σ -15,t )+β 1.4. log(σ -30,t )+u 3,t ;

[0091] log(σ 7,t )=α7+β 2,1 log(σ -3,t )+β 2,2 log(σ -7,t )+β 2.3 log(σ -15,t )+β 2.4. log(σ -30,t )+u 7,t ;

[0092] log(σ 15,t )=α 15 +β 3,1 log(σ -3,t )+β 3,2 log(σ -7,t )+β 3.3 log(σ -15,t )+β 3.4. log(σ -30,t )+u 15,t ;

[0093] log(σ 30,t )=α 30 +β 4,1 log(σ -3,t )+β 4,2 log(σ -7,t )+β 4.3 log(σ -15,t )+β 4.4. log(σ -30,t )+u 30,t ;

[0094] σ z,t This represents the volatility of the target company's stock price over the days z = 3, 7, 15, and 30, where z ≤ F and z ∈ t; u z,t It is a white noise term; β a,b It is a coefficient matrix of dynamic relationships, a,b=1,2,3,4; α z This is the intercept term; stock price volatility is defined as the standard deviation of the target company's returns over z days: The number of days z here is 3, 7, 15 and 30, which are just examples and not a limitation on the scheme itself.

[0095] Through this step S5, the time-series transaction data is converted into a high-dimensional feature vector T. v The parameters related to dynamic relationships provide key time-series information and dynamic relationship support for financial risk prediction.

[0096] S6: The obtained audio feature vector, text summary feature vector, embedding vector corresponding to the summary of the text summary, news text feature vector, and time series data feature vector are fused to obtain a joint representation vector.

[0097] The joint representation vector E mentioned in step S6 is fused using the following formula: E = w0 + w1T a +w2T t +w3T l +w4T n +w5T v +ε, where w0 is the bias term, w1, w2, w3, w4, w5 are the fusion weights, and ε is the error term, representing random noise.

[0098] The joint representation vector E has a fixed dimension of 512, representing the comprehensive features of the multimodal data. This vector retains important information from each modality's features and can be used for subsequent multi-task prediction.

[0099] S7: Input the joint representation vector into the multi-task learning framework to predict risk indicators.

[0100] Specifically, the joint representation vector E generated in step S6 is used as the input to the multi-task learning framework, which simultaneously predicts the following risk indicator: the volatility σ of the target company's stock price over time spans of 3 days, 7 days, 15 days, and 30 days. 3,t σ 7,t σ 15,t and σ 30,t And the daily risk value VAR;

[0101] For each time span, an independent first prediction subnetwork (MLP) is constructed to predict stock price volatility. The prediction results for stock price volatility are as follows: in f represents the predicted volatility. MLP,z (·) represents the first prediction subnetwork MLP with a time span of z;

[0102] For the daily risk value (VAR), an independent second prediction subnetwork (MLP) is constructed, with the joint representation vector E as input and the predicted VAR value as output. in f is the predicted value of VAR; MLP,VAR (·) represents the second prediction subnetwork MLP;

[0103] The joint loss function simultaneously optimizes both stock price volatility and VAR prediction tasks. The formula for the joint loss function is: μ is a weighted hyperparameter that balances the prediction error of stock price volatility and the prediction error of VAR; y j and It represents the true and predicted values ​​of the stock price volatility indicator; The mean squared error of stock price volatility forecast; q represents the quantile threshold of daily risk value; V and These represent the actual daily risk value and the predicted daily risk value, respectively. This represents the quantile regression loss function for the daily risk value prediction task.

[0104] Through step S7 above, the multi-task learning framework can effectively integrate multimodal features, accurately predict volatility and daily risk values ​​over different time spans, and provide reliable support for financial risk management.

[0105] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A financial risk prediction method based on multimodal data fusion, characterized in that, Includes the following steps: S1: Extract audio feature vectors from the target company's earnings call; The specific content of step S1 is as follows: S11: Use the WenetSpeech pre-trained language model to extract the embedding vectors from the earnings call audio. For a pre-processed audio segment... in Let i represent the i-th frame of the audio, and n represent the number of frames in the audio. Convert each frame of audio into a vector representation. Thus, the entire audio segment A is obtained. c audio embedding vector S12: Embed the audio vector E ac The audio feature vector T is then fed into a multi-head self-attention module (MHSA) to further extract the audio feature vector T. ac =MHSA(E ac ); S13: Transfer the feature vector T ac The compressed audio feature vector T is obtained by feeding it into an average pooling layer. a T a =AveragePooling(T ac ); S2: Extract feature vectors from the text transcripts of the target company's earnings call; S3: Use a large language model to summarize the text transcripts of the target company's earnings call and obtain the embedding vector corresponding to the summary of the text transcripts. S4: Extract news text features from the target company to obtain news text feature vectors; S5: Extract features from the time-series transaction data of the target company during a period before the target date to obtain the feature vector of the time-series data; S6: Fuse the obtained audio feature vector, text summary feature vector, embedding vector corresponding to the text summary, news text feature vector, and time-series data feature vector to obtain a joint representation vector; and S7: Input the joint representation vector into the multi-task learning framework to predict risk indicators.

2. The financial risk prediction method based on multimodal data fusion according to claim 1, characterized in that, Vector representation of each frame of audio All dimensions are 512, and the compressed audio feature vector T a The dimension of the text summary feature vector is 512; the dimension of the text summary feature vector is 768; the dimension of the embedding vector corresponding to the summary of the text summary is 768; and the dimension of the joint representation vector is 512.

3. The financial risk prediction method based on multimodal data fusion according to claim 1, characterized in that, The Multi-Head Self-Attention Module (MHSA) processes the input audio embedding vector in parallel using multiple attention heads, and calculates the attention weights of the audio embedding vector using the following formula: Where the query vector Q = E ac W Q The key vector K = E ac W K Value vector V = E ac W V W Q W K and W V Let d be a linear projection matrix. k The dimension of the attention heads is defined, and the softmax function is used for normalization. The outputs of each attention head are concatenated, and a linear transformation is applied to obtain the audio feature vector T. ac T ac =Concat(head1,head2,...head) h W o Where subscripts 1, 2, ..., h are distinguishing markers for different attention heads, W o Let be the linear transformation matrix after merging, and Concat be the merging function.

4. The financial risk prediction method based on multimodal data fusion according to claim 3, characterized in that, The specific content of step S2 is as follows: S21: Preprocess the transcript of the target company's earnings call to obtain a set of sentences. in Let L represent the L-th sentence in the text summary, where L = 1, 2, ..., l, and l represents the number of sentences in the text summary; using the Sentence-BERT pre-trained language model, each sentence... Mapped to the embedding vector of the earnings call transcript Obtain the vector representation of the entire earnings call transcript. S22: The vector representation of the entire earnings call transcript is fed into the multi-head self-attention module MHSA to further extract the feature vector T of the transcript. tc T tc =MHSA(E tc ); S23: Transfer the feature vector T of the text summary tc The data is fed into an Average Pooling Layer to obtain the compressed text summary feature vector T. t =AveragePooling(T tc ).

5. The financial risk prediction method based on multimodal data fusion according to claim 4, characterized in that, The specific content of step S3 is as follows: S31: Paragraph segmentation. The transcript of the target company's earnings call is segmented into logical paragraphs to obtain paragraph p. M =p1,p2,...,p m M = 1, 2, ..., m, where the subscript m represents the number of paragraphs; for each paragraph p M Input a large language model (LLM) to extract paragraph-level summaries. M =LLM(p M ), s M It is paragraph p M Abstract; S32: Summarize all paragraphs {s1,s2,...s} M The data is merged into a single text and input into a large language model to generate a comprehensive summary S, where S = LLM({s1,s2,...s...). M }); S33: Use the Sentence-BERT pre-trained language model to vectorize the comprehensive summary S, generating the corresponding embedding vector T for the summary of the text minutes. l T l =SBERT(S).

6. The financial risk prediction method based on multimodal data fusion according to claim 5, characterized in that, The specific content of step S4 is as follows: S41: Collect news text data of the target company several days before the target transaction date, represented as N = {n1, n2, ... n} k }, where n K0 This represents the K0th news article, where k is the total number of news articles. S42: Use a Large Language Model (LLM) for each news item n K0 Perform parsing and extract metadata m K0 m K0 =LLM(n K0 ); Integrate the metadata of all news articles into a single metadata set M. N ={m1,m2,...,m k }; S43: Find the k historical news groups N in the historical news dataset that are most similar to the news text data N from several days prior to the target transaction date in terms of metadata. K0 =N1,N2,...,N k Calculate the relationship between N and any set of historical news groups N K0 semantic relevance, Where f is a function that converts news text into an embedded vector, and selects a set of historical news H that has the highest semantic similarity to news text data N from several days before the target trading day; S44: Concatenate the text related to N, H, and subsequent market trends after H, and then use the SBERT model to convert the concatenated text into an embedding vector T. n Embedded vector T n This serves as a feature vector of news texts from the target company several days prior to the target transaction date.

7. A financial risk prediction method based on multimodal data fusion according to claim 6, characterized in that, The specific content of step S5 is as follows: S51: Collect the target company's time-series transaction data D for the 30 days prior to the target date, including daily closing prices and trading volumes, represented as D = {(p1,v1),(p2,v2),...,(p...}. d ,v d )}, where p F =p1,p2,...,p d v represents the closing price on day F. F =v1,v2,...,v d This represents the trading volume on day F. S52: Input the time-series transaction data D into the Bi-Short Memory (Bi-LSTM) network. The Bi-LSTM network captures the temporal dynamics of the transaction data and outputs a feature vector T containing the time-series data. v T v =BiLSTM(D); S53: Capturing the dynamic relationships between different time-series features using a vector autoregressive (VAR) model: log(σ 3,t )=α3+β 1,1 log(σ -3,t )+b 1,2 log(σ -7,t )+b 1.3 log(σ -15,t )+b 1.

4. log(σ -30,t )+u 3,t ; log(σ 7,t )=α7+β 2,1 log(σ -3,t )+b 2,2 log(σ -7,t )+b 2.3 log(σ -15,t )+b 2.

4. log(σ -30,t )+u 7,t ; log(σ 15,t )=a 15 +b 3,1 log(σ -3,t )+b 3,2 log(σ -7,t )+b 3.3 log(σ -15,t )+b 3.

4. log(σ -30,t )+u 15,t ; log(σ 30,t )=a 30 +b 4,1 log(σ -3,t )+b 4,2 log(σ -7,t )+b 4.3 log(σ -15,t )+b 4.

4. log(σ -30,t )+u 30,t ; σ z,t This represents the volatility of the target company's stock price over the days z = 3, 7, 15, and 30, where z ≤ F and z ∈ t; u z,t It is a white noise term; β a,b It is a coefficient matrix of dynamic relationships, a,b=1,2,3,4; α z This is the intercept term; stock price volatility is defined as the standard deviation of the target company's returns over z days:

8. A financial risk prediction method based on multimodal data fusion according to claim 7, characterized in that, The joint representation vector E mentioned in step S6 is fused using the following formula: E = w0 + w1T a +w2T t +w3T l +w4T n +w5T v +ε, where w0 is the bias term, w1, w2, w3, w4, w5 are the fusion weights, and ε is the error term, representing random noise.

9. A financial risk prediction method based on multimodal data fusion according to claim 8, characterized in that, Step S7 involves using the joint representation vector E generated in step S6 as input to a multi-task learning framework. This framework simultaneously predicts the following risk indicator: the volatility σ of the target company's stock price over time spans of 3, 7, 15, and 30 days. 3,t σ 7,t σ 15,t and σ 30,t And the daily risk value VAR; For each time span, an independent first prediction subnetwork (MLP) is constructed to predict stock price volatility. The prediction results for stock price volatility are as follows: in f represents the predicted volatility. MLP,z (·) represents the first prediction subnetwork MLP with a time span of z; For the daily risk value (VAR), an independent second prediction subnetwork (MLP) is constructed, with the joint representation vector E as input and the predicted VAR value as output. in f is the predicted value of VAR; MLP,VAR (·) represents the second prediction subnetwork MLP; The joint loss function simultaneously optimizes both stock price volatility and VAR prediction tasks. The formula for the joint loss function is: μ is a weighted hyperparameter that balances the prediction error of stock price volatility and the prediction error of VAR; y j and It represents the true and predicted values ​​of the stock price volatility indicator; The mean squared error of stock price volatility forecast; q represents the quantile threshold of daily risk value; V and These represent the actual daily risk value and the predicted daily risk value, respectively. This represents the quantile regression loss function for the daily risk value prediction task.

Images