Financial public opinion sentiment analysis method based on multi-modal dynamic weight fusion

Abstract

The invention relates to the field of public opinion analysis, in particular to a financial public opinion sentiment analysis method based on multi-modal dynamic weight fusion, and the method comprises the steps: obtaining multi-modal data input in a financial public opinion scene, and carrying out the feature extraction of different modal data input; based on a self-adaptive attention fusion mechanism, dynamically calculating emotional contribution weights of different modes in a current financial scene, and performing weighted fusion on multi-mode input; classifying the multi-modal features by adopting an emotion classifier to obtain emotion polarity and a continuous emotion intensity value; performing abnormal conflict detection and secondary verification on different modal inputs; and performing dynamic confidence coefficient calibration, and writing the emotional intensity value into a time sequence database in real time. According to the financial public opinion sentiment analysis method, the judgment accuracy and robustness of the financial public opinion sentiment analysis method are comprehensively improved through the design of pre-construction of the financial scene template library, the abnormal conflict detection and secondary verification mechanism, the time attenuation factor and the behavior confidence coefficient calibration mechanism.

Description

Technical Field

[0001] This application relates to the field of public opinion analysis, specifically to a financial public opinion sentiment analysis method based on multimodal dynamic weight fusion. Background Technology

[0002] With the rapid development of the internet and social media, the speed of information dissemination in financial markets has accelerated significantly, and the amount of information has increased dramatically. This information includes not only traditional news reports and company announcements, but also a large amount of multimodal data from social media platforms, such as user comments, memes, and charts. In this complex information environment, accurately capturing public sentiment is crucial for risk warning, investment decisions, and regulatory compliance.

[0003] In existing technologies, traditional financial sentiment analysis mainly relies on single-modal data processing methods. However, in different scenarios, especially when dealing with content containing rich visual elements or unstructured expressions, sentiment data often contains noisy information such as irony, sarcasm, or multimodal contradictions, which affects the judgment results of sentiment analysis. In addition, traditional financial sentiment analysis often ignores the historical evolution trend of sentiment.

[0004] Therefore, there is an urgent need for a financial sentiment analysis method that can integrate multi-source information and effectively handle contradictory noise in multimodal data. Summary of the Invention

[0005] In view of this, this application discloses a financial sentiment analysis method based on multimodal dynamic weight fusion to solve the problems in the prior art, including:

[0006] S1. Obtain multimodal data input in the context of financial public opinion, and perform standardized preprocessing on the multimodal data input;

[0007] S2. Perform feature extraction on the data inputs of different modalities to obtain the feature inputs of different modalities;

[0008] S3. Based on the adaptive attention fusion mechanism, the emotional contribution weights of different modalities in the current financial scenario are dynamically calculated, and the multimodal inputs are weighted and fused to obtain multimodal features. An emotional classifier is used to classify the multimodal features to obtain emotional polarity and continuous emotional intensity values, and interpretable evidence is generated.

[0009] S4. Perform abnormal conflict detection and secondary verification on different modal inputs; the abnormal conflict detection is performed by calculating the cosine similarity between the text sentiment score and the image sentiment embedding vector to determine the consistency between modalities; if the similarity is lower than the preset similarity threshold, it is determined to be a modal conflict and triggers the secondary verification mechanism; the secondary verification is based on the historical behavior statistics and the current confidence difference of each modality for contextual verification.

[0010] S5. Perform dynamic confidence calibration on the emotional intensity value and write it to the time series database in real time.

[0011] The beneficial effects of this application include:

[0012] By pre-building a financial scenario template library, the system dynamically and adaptively fuses text, image, and metadata modal information according to different financial scenarios. This captures complex signals such as chart trends, emoticons, and sarcastic contexts in financial public opinion, improving the accuracy and granularity of sentiment polarity and intensity recognition. Furthermore, by combining contextual features such as content type, source platform, and time sensitivity, the system automatically adjusts the contribution weight of each modality in different financial scenarios, achieving "context-specific" intelligent sentiment assessment and enhancing generalization ability and practicality.

[0013] An abnormal conflict detection and secondary verification mechanism is designed. By quantifying the consistency of sentiment between text and image, it identifies and processes high-noise situations such as irony, sarcasm, or multimodal contradictions. Secondary verification is performed based on historical behavior statistics and confidence differences, thereby effectively reducing the false judgment rate and improving the robustness of sentiment analysis in real and complex environments.

[0014] By integrating the time decay factor and the behavioral confidence calibration mechanism, sentiment judgment not only reflects the current input but also relates to the historical trend of public opinion evolution, avoiding bias caused by isolated evaluation; at the same time, it supports dynamic compression or amplification of sentiment intensity to ensure that the output results are more realistically interpretable. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the financial sentiment analysis method based on multimodal dynamic weight fusion in Embodiment 1 of this application.

[0016] Figure 2 This is a schematic diagram of the neural network model in Embodiment 2 of this application;

[0017] Figure 3 This is a schematic diagram of the dynamic weight fusion module in Embodiment 2 of this application;

[0018] Figure 4 This is a flowchart illustrating the abnormal conflict detection module in Embodiment 2 of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, features, and advantages of this application clearer and to enable those skilled in the art to better understand the technical solutions of this application, the following detailed description of this application is provided in conjunction with the accompanying drawings and embodiments.

[0020] Example 1:

[0021] This embodiment includes a financial sentiment analysis method based on multimodal dynamic weight fusion, such as... Figure 1 As shown, it includes:

[0022] S1. Obtain multimodal data input in a financial public opinion scenario, and perform standardized preprocessing on the multimodal data input. The multimodal data input includes:

[0023] Text data: such as financial news, social media comments, company announcements, etc.;

[0024] Image data: such as candlestick charts, financial report screenshots, press conference photos, and emotional emojis;

[0025] Metadata includes content type, source platform, publication time, and user influence level.

[0026] Text data, image data, and metadata are all financial public opinion information to be analyzed.

[0027] S2. Perform feature extraction on the different modal data inputs to obtain the feature inputs of different modalities.

[0028] For text modalities, semantic embedding vectors are generated using a language model fine-tuned for the financial domain. The formula is:

[0029]

[0030] For image modalities, high-level visual semantic features are extracted using a Visual Transformer (ViT). It is specifically optimized for recognizing financial charts and sentiment symbols. The formula is as follows:

[0031]

[0032] For metadata, the metadata features are obtained by performing structured encoding on the metadata. The formula is:

[0033]

[0034] in, Indicates the content type. Indicates platform, Representing the publication timestamp, discrete metadata is embedded through a layer. and Mapped to vector form and associated with continuous fields The components are then concatenated and MLP encoded.

[0035] S3. Based on an adaptive attention fusion mechanism, the emotional contribution weights of different modalities in the current financial scenario are dynamically calculated, and the multimodal inputs are weighted and fused to obtain multimodal features. An emotional classifier is then used to classify the multimodal features to obtain emotional polarity and continuous emotional intensity values. And generate interpretable evidence. This includes:

[0036] S31. Calculate the semantic relevance score between text and image to generate initial weights for cross-modal attention; the formula is:

[0037]

[0038]

[0039]

[0040] in, Indicates the initial weights for cross-modal attention. The text-image semantic relevance score is represented. The image-text semantic relevance score represents the image-text semantic relevance score. Represents the semantic embedding vector of the text. and Represents the learnable parameter matrix, Represents high-level visual semantic features. Indicates the embedding dimension.

[0041] S32. Construct a financial scenario template library; match the corresponding financial scenario template library according to the content type and platform source of the metadata.

[0042] The financial scenario template library contains different financial scenarios. constitute, ; , , This represents the perception weighting factor for different modalities within the same scenario. When constructing the financial scenario template library, the scenario type for each data point is labeled, and a perception weighting factor value is set for different financial scenarios r. The perception weighting factor value differs in different scenarios, representing different modal priority strategies. For example, in the "financial report release" scenario... Relatively high; under "Retail Investor Discussion", and Dominant; .

[0043] S33. An exponential decay function is used to assign decreasing weights to the fusion behavior, i.e., a time sensitivity factor is set. Multimodal inputs are weighted and fused to ensure that recent public opinion has higher priority for current judgments; the formula for the time sensitivity factor is:

[0044]

[0045] in, Indicates the decay rate. Indicates the time when the event occurred. Indicates the current moment.

[0046] Furthermore, the final weights for each mode are:

[0047]

[0048]

[0049]

[0050] in, , , These represent the relative fusion weights of text semantic embedding vectors, visual semantic features, and metadata features, respectively. , , These represent text-aware weighting factors, visual-aware weighting factors, and metadata-aware weighting factors, respectively, taken from a financial scenario template library. , representing the adaptive balance coefficient, yields the weighted multimodal features. The formula is:

[0051]

[0052] in, Represents multimodal features, , , These represent text semantic embedding vectors, visual semantic features, and metadata features, respectively.

[0053] S34. A sentiment classifier is used to process the multimodal features to obtain sentiment polarity and continuous sentiment intensity values. And generate interpretable evidence.

[0054] S4. Perform abnormal conflict detection and secondary verification on different modal inputs.

[0055] This application designs an anomaly conflict detection and secondary verification mechanism to handle the presence of irony, sarcasm, and other satirical content in multimodal data input; the anomaly conflict detection is achieved by calculating the text sentiment score. With image emotion embedding vector The cosine similarity between modalities is used to determine the consistency between modalities. If the similarity is lower than the preset similarity threshold, it is determined to be a modal conflict and a secondary verification mechanism is triggered. The secondary verification is based on the difference between historical behavior statistics and the current confidence of each modality to perform contextual verification, avoiding reliance on external knowledge graphs and making judgments entirely by the internal logic of the model.

[0056] Specifically, the formula for calculating the cosine similarity between the text sentiment score and the image sentiment embedding vector is as follows:

[0057]

[0058]

[0059]

[0060] in, express and cosine similarity, Indicates a text encoder. This represents the learnable weight parameters; in this embodiment, the preset similarity threshold is θ=0.3.

[0061] The secondary verification includes:

[0062] Step 1: Calculate the current confidence level for each modality.

[0063] For text modality: obtain the sentiment classification probability distribution based on the output of the fine-tuned language model in the financial field, and select the probability of the largest category as the text sentiment confidence;

[0064] For image modalities: Based on the image sentiment embedding vector extracted by the visual Transformer, the probability distribution output by the image sentiment classification head is used to take the probability of the dominant sentiment category as the image sentiment confidence.

[0065] Step 2: Assess the difference in confidence between modalities; calculate the absolute difference in confidence between the text and image modalities: if the confidence of one modality is significantly higher than that of another modality, then the sentiment judgment of the higher confidence modality is adopted.

[0066] Step 3: Perform fusion strategy recalibration.

[0067] After confirming the conflict, instead of directly using the original fusion results, the weight of the low-confidence modality is reduced or the contradictory modality is temporarily masked, and only the high-confidence modality is used to output the initial sentiment.

[0068] Step 4: Generate interpretability evidence; record the reasons for triggering secondary verification, the confidence level of each modality, historical behavior reference evidence, etc., for subsequent auditing or manual review.

[0069] S5. Perform dynamic confidence calibration on the emotional intensity value and write it to the time series database in real time.

[0070] The dynamic confidence calibration is achieved by calculating the modal fusion entropy. Compared to historical prediction accuracy, the sentiment intensity value is compressed or amplified to improve the reliability of the results; the formula for dynamically calibrating the sentiment intensity confidence level is:

[0071]

[0072]

[0073] in, A calibration value representing the intensity of emotion. A parameter representing the balance between entropy and accuracy. Represents modal fusion entropy, used to measure modal consistency. Indicates the historical prediction accuracy rate. This represents the relative fusion weights of different modalities.

[0074] Furthermore, the emotional intensity value is written into the time-series database in real time. When a high-risk negative emotional cluster is detected, a three-level automatic early warning signal is triggered based on the emotional intensity, the speed of spread, and the importance of the financial entities involved.

[0075] Example 2:

[0076] This embodiment includes a financial sentiment analysis method based on multimodal dynamic weight fusion. The difference from Embodiment 1 is that the financial sentiment analysis method in this embodiment is implemented based on a neural network model, as shown in the example below. Figure 2 As shown, it includes: a text encoder, an image encoder, a metadata embedding unit, a dynamic weight fusion module, a sentiment decision-making module, an anomaly / conflict detection module, and a result storage and early warning unit; the dynamic weight fusion module is as follows: Figure 3 As shown, the flowchart of the anomaly conflict detection module is as follows: Figure 4 As shown.

[0077] Finally, it should be noted that the above description only depicts some embodiments of this application. For those skilled in the art, various changes, modifications, substitutions, and variations can be conceived of these embodiments without departing from the principles and spirit of this application. The scope of protection of this application is defined by the appended claims and their equivalents, and all the above-mentioned behaviors should be covered within the scope of protection of this application.

Claims

1. A financial sentiment analysis method based on multimodal dynamic weight fusion, characterized in that, include: S1. Obtain multimodal data input in the context of financial public opinion, and perform standardized preprocessing on the multimodal data input; The multimodal data input includes: text data, image data, and metadata; S2. Perform feature extraction on the data inputs of different modalities to obtain the feature inputs of different modalities; S3. Based on the adaptive attention fusion mechanism, the emotional contribution weights of different modalities in the current financial scenario are dynamically calculated, and the multimodal inputs are weighted and fused to obtain multimodal features. An emotional classifier is used to classify the multimodal features to obtain emotional polarity and continuous emotional intensity values, and interpretable evidence is generated. S4. Perform abnormal conflict detection and secondary verification on different modal inputs; the abnormal conflict detection is performed by calculating the cosine similarity between the text sentiment score and the image sentiment embedding vector to determine the consistency between modalities; if the similarity is lower than the preset similarity threshold, it is determined to be a modal conflict and triggers the secondary verification mechanism; the secondary verification is based on the historical behavior statistics and the current confidence difference of each modality for contextual verification. S5. Perform dynamic confidence calibration on the emotional intensity value and write it to the time series database in real time.

2. The financial sentiment analysis method based on multimodal dynamic weight fusion according to claim 1, characterized in that, The process of obtaining emotional polarity and continuous emotional intensity values ​​includes: S31. Calculate the semantic relevance score between text and image, and generate initial weights for cross-modal attention; S32. Construct a financial scenario template library; match the corresponding financial scenario template library according to the content type and platform source of the metadata; S33. An exponential decay function is used to assign decreasing weights to the fusion behavior, and weighted fusion of multimodal inputs is performed. S34. Use an emotion classifier to process multimodal features, obtain emotion polarity and continuous emotion intensity values, and generate interpretable evidence.

3. The financial sentiment analysis method based on multimodal dynamic weight fusion according to claim 2, characterized in that, The financial scenario template library contains different financial scenarios. constitute, ; , , Indicating financial scenarios Different perceptual weight factors are assigned to different modalities. When constructing a financial scenario template library, the scenario type of each data point is labeled, and perceptual weight factor values ​​are set for different financial scenarios. The perceptual weight factor values ​​are different for different scenarios.

4. The financial sentiment analysis method based on multimodal dynamic weight fusion according to claim 3, characterized in that, The financial scenario , .

5. The financial sentiment analysis method based on multimodal dynamic weight fusion according to claim 2, characterized in that, The process employs an exponential decay function to assign decreasing weights to the fusion behavior and sets a time-sensitivity factor. The formula for the time sensitivity factor in the weighted fusion of multimodal inputs is: ； in, Indicates the decay rate. Indicates the time when the event occurred. Indicates the current moment.

6. The financial sentiment analysis method based on multimodal dynamic weight fusion according to claim 5, characterized in that, The setting of time sensitivity factor The weighted fusion of multimodal inputs is performed using the following formula: ； in, Represents multimodal features, , , These represent text semantic embedding vectors, visual semantic features, and metadata features, respectively. , , These represent the relative fusion weights of text semantic embedding vectors, visual semantic features, and metadata features, respectively.

7. The financial sentiment analysis method based on multimodal dynamic weight fusion according to claim 6, characterized in that, and The formula is: ； ； in, Indicates the adaptive balance coefficient. Indicates the initial weights for cross-modal attention. and These represent the text perception weight factor and the visual perception weight factor, respectively.

8. The financial sentiment analysis method based on multimodal dynamic weight fusion according to claim 1, characterized in that, The secondary verification includes: Step 1: Calculate the current confidence level for each modality; Step 2: Assess the confidence differences between modalities; Step 3: Perform a fusion strategy recalibration; reduce the weight of low-confidence modalities or temporarily shield contradictory modalities, relying only on high-confidence modalities to output preliminary sentiment. Step 4: Generate interpretability evidence; record the reasons for triggering secondary verification, the confidence level of each modality, historical behavior reference evidence, etc., for subsequent auditing or manual review.

9. The financial sentiment analysis method based on multimodal dynamic weight fusion according to claim 1, characterized in that, The dynamic confidence calibration is achieved by calculating the modal fusion entropy. Compared with historical prediction accuracy, the emotional intensity value Compress or amplify; The formula for dynamically calibrating the confidence level of emotional intensity is: ； ； in, A calibration value representing the intensity of emotion. A parameter representing the balance between entropy and accuracy. Represents the modal fusion entropy, Indicates the historical prediction accuracy rate. This represents the relative fusion weights of different modalities.

Images